Outages and Maintenance
-
Standby queues paused on Hansen Cluster
Update As of 5:20pm the standby and standby-c queues have been started and their jobs are being scheduled for execution. The standby queues on Hansen have been paused temporarily in order to address a slow scheduling issue. We expect to have them ba...
-
Unscheduled Scratch Outage on Carter
The scratch storage on Carter and Scholar has been returned to normal operations. The rebuild process will be continuing in the background, so we will be watching for any degradation in the storage performance. All queues have been re-activated. T...
-
Unscheduled Scheduling Outage on Hansen
Job scheduling on Hansen has returned to normal. This concludes the outage. Original Message: Hansen is not currently scheduling any new jobs. A filesystem used for a specific project serving Hansen became unavailable on Hansen around 5:00am. This...
-
Unscheduled Scratch Outage on Carter
UPDATE: The issue with Carter's scratch filesystem has been resolved. The filesystem is now available. Job scheduling on the cluster has been resumed. The scratch filesystem serving Carter is currently unavailable. Job scheduling on Carter has bee...
-
Conte has been returned to normal operations as of Wednesday, May 18th, 2016 at 5:55am EDT. All upgrades were completed, though a small number of nodes which require more attention to be fully ready for jobs remain offline for now and will be return...
-
The Isilon filesystem was restored to normal service and all affected clusters had it remounted as quickly as was sustainable by the filesystem. This process was completed by Wednesday, May 18th, 2016 at 12:15am EDT. All clusters other than Conte (...
-
Network issues affecting Snyder cluster
The problem is now RESOLVED after the reboot of a router. ======= The network serving Snyder is currently experiencing issues. Attempts to log in to the cluster or access network filesystems will experience high latency and delays. Systems engineers...
-
Networking to and from campus, and around large parts of campus are down. Many services are unreachable at the moment. We will provide updates as they become available.
-
Networking to and from campus, and around large parts of campus are down. Many services are unreachable at the moment. We will provide updates as they become available.
-
Carter and Scholar are back online for use as of 6:25am, though they will be operating with many nodes still offline. Staff will be working through Wednesday to steadily increase the number of nodes available. This concludes the POD cluster mainten...
-
An issue was discovered shortly after Conte, Hansen, Hathi, and Radon were brought back online with the /group path on several front-ends and nodes. Any scripts or jobs that rely on the /group path may have had issues immediately following return to...
-
The Depot maintenance has been completed successfully. Depot is now back to normal operations. Other compute cluster outages running concurrent with this however, are still in progress. This maintenance window next week has been reduced as much as...
-
The underlying storage has been fixed, and all these clusters have been returned to normal operations as of 10:00pm EDT. As of Tuesday, June 7th, 2016 at 4:10pm EDT, Conte, Hansen, Hathi, and Radon are unavailable due to a loss of Isilon home direct...
-
The maintenance is complete and github.rcac is back in production, now at version 2.6.4. There are several bug fixes and security patches in this version, but no major feature updates. Additionally, an infrastructure change has been made to the way...
-
The maintenance is complete and github.rcac is back in production, now at version 2.6.4. There are several bug fixes and security patches in this version, but no major feature updates. Additionally, an infrastructure change has been made to the way...
-
Engineering Computing Network (ECN) will be performing scheduled maintenance this weekend on several ECN server resulting in their unavailability for a short time. Some ECN services will be affected, including several software license servers for ITa...
-
Unscheduled Outage on Data Depot
As of 7:30 pm, all methods for connecting to Data Depot have been restored to working order. All connections with Samba (Network Drive mappings: datadepot.rcac.purdue.edu, samba.rcac.purdue.edu) are working normally again. More Rice and Snyder nodes...
-
Self-service management web tool outage
As of 3:20 pm, the self-service tool is back in action. An issue with the database backing authentication was discovered and repaired. Original message The self-service management tool (user management) is experiencing issues with authentication. Att...
-
The monthly GitHub maintenance is now complete and github.rcac is back up and running with the latest bug fixes and security patches. Thank you for your patience. Please let us know if you see any issues at rcac-help@purdue.edu. Original Message Once...
-
Unscheduled Outage on Data Depot
UPDATE As of 5:30 pm. Friday, 5 August, 2016, we believe the problem affecting access to the Data Depot has been corrected. Thank you for your patience, and I apologize for the disruption this caused. Original Message Access to the Data Depot is curr...