<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:media="http://search.yahoo.com/mrss/">
	<channel>
		<title>RCAC - Outages</title>
		<link>https://rcac.purdue.edu/index.php/news/rss/Outages</link>
		<description><![CDATA[news::news.feed description]]></description>
		<atom:link href="https://rcac.purdue.edu/index.php/news/rss/Outages" rel="self" type="application/rss+xml" />
		<language>en</language>
		<lastBuildDate>Tue, 07 Apr 2026 08:56:55 EDT</lastBuildDate>
					<item>
				<title><![CDATA[Negishi Cluster Filesystem Interruption (Resolved)]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7651</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7651</guid>
				<description><![CDATA[<p>Between approximately 9:30 PM and 10:45 PM EDT on March 30, 2026, the Negishi home file system experienced issues that prevented users from successfully connecting to login nodes.</p>
<p>Service was fully restored at 10:45 PM EDT, and login functionality has returned to normal. No data loss occurred.</p>
]]></description>
				<pubDate>Mon, 30 Mar 2026 21:30:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Temporary Impact to Data Depot Operations]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7650</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7650</guid>
				<description><![CDATA[<p>We’re currently investigating an issue affecting IO operations to the Data Depot. Our monitoring has detected that the primary storage disks used for data writes are nearing capacity, which is causing IO failures for jobs writing to the Depot.</p>
<p>To prevent further impact, job submissions to clusters connected to Depot have been temporarily paused while the team works restore normal operation.</p>
<p><strong>Impact:</strong> Users may experience job delays or failures when writing to the Data Depot.</p>
<p><strong>Current Status:</strong> Our engineering team is actively addressing the capacity issue.</p>
<p><strong>Next Update:</strong> We will provide an update once the service is restored or we have more information to share.</p>
]]></description>
				<pubDate>Mon, 30 Mar 2026 09:30:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Power Outage Impacting Multiple Clusters — Recovery Underway]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7640</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7640</guid>
				<description><![CDATA[<p>At approximately 6:00 AM EDT, a power outage impacted systems in the Math Data Center. Most services have now been restored.</p>
<p>Due to the outage, some jobs on Gilbreth did not requeue automatically. Users should check the status of any jobs that were running early this morning and resubmit them if needed.</p>
]]></description>
				<pubDate>Wed, 18 Mar 2026 06:00:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Negishi Cluster Service Interruption]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7629</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7629</guid>
				<description><![CDATA[<p>We are currently experiencing an unexpected outage affecting the Negishi cluster. We are actively working to restore service and will provide an update once the system is stable.</p>
<ul>
<li>
<p><strong>Impact:</strong> Users are currently unable to access their home directories, and SSH connections or active terminal sessions may freeze or be denied.</p>
</li>
<li>
<p><strong>Current action:</strong> System administrators are performing a system reboot of the affected storage infrastructure to clear the error and restore file access.</p>
</li>
</ul>
]]></description>
				<pubDate>Wed, 11 Mar 2026 15:00:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[MATH Datacenter Cooling issue - Job scheduling paused on Anvil/Gautschi]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7625</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7625</guid>
				<description><![CDATA[<p>The MATH datacenter started experiencing issues with cooling systems around 12pm. Job scheduling on the Anvil and Gautschi clusters was paused shortly after and scheduling resumed at 1:30pm.</p>
]]></description>
				<pubDate>Wed, 04 Mar 2026 12:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Data Depot Filesystem issue: Scheduling Resumed]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7611</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7611</guid>
				<description><![CDATA[<p>An internal portion of the Data Depot filesystem is currently offline, as a result, all scheduling has been paused until this issue is resolved.</p>
<p><strong>Impact to you</strong>
Attempts to read files that are on the affected storage may result in error messages</p>
<p>Our IT team is actively working with the vendor to restore service as quickly as possible. We will send an update as soon as more information is available.</p>
]]></description>
				<pubDate>Wed, 11 Feb 2026 14:30:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Gilbreth Remains Unavailable – Update Expected by 10:00 AM]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7608</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7608</guid>
				<description><![CDATA[<p>The scheduled Math Data Center maintenance is complete. The <strong>Gilbreth cluster remains unavailable</strong> as we continue post‑maintenance work. <strong>The next update will be provided by 10:00 AM tomorrow (Friday), or sooner if available.</strong></p>
<p>We appreciate your patience and understanding as we work to restore Gilbreth service.</p>
]]></description>
				<pubDate>Fri, 06 Feb 2026 00:15:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Data Depot Service Outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7607</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7607</guid>
				<description><![CDATA[<p>RCAC’s Data Depot storage system is currently unavailable. Our team is actively investigating and working to restore service as quickly as possible.</p>
<p>We will provide updates as the situation develops and once service is fully restored.</p>
<p>We appreciate your patience and understanding.</p>
]]></description>
				<pubDate>Thu, 05 Feb 2026 08:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Globus access to Depot degraded; slow Depot logins and Depot access on clusters]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7605</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7605</guid>
				<description><![CDATA[<p>Users of Data Depot on RCAC clusters are currently experiencing degraded performance, and some Globus transfers to and from Depot are failing or running slowly.  In addition, some users may see slow Globus logins or be temporarily unable to log in to Globus when accessing Depot collections.</p>
<p>System monitoring has identified an issue where heavy job activity was overloading the Data Depot filesystem used by the clusters and Globus.</p>
<p>You may see the following impacts:</p>
<ul>
<li>Globus transfers to and from Depot collections may fail, stall, or run much more slowly than usual.</li>
<li>Globus logins may be slow or occasionally fail when accessing Depot endpoints.</li>
<li>Jobs on RCAC clusters that read from or write to Depot may experience slow file access, delayed directory listings, or timeouts.</li>
</ul>
<p>Our engineers are investigating the high load from a large number of concurrent jobs and are working to reduce the impact on Depot, Globus, and cluster workloads.  Existing jobs will continue to run, but any that are heavily Depot‑I/O‑bound may run more slowly or see I/O errors until performance improves.  We will provide another update by 5:00PM EST or sooner if the issue is resolved.</p>
]]></description>
				<pubDate>Fri, 30 Jan 2026 15:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Geddes Storage Outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7603</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7603</guid>
				<description><![CDATA[<p>The storage system for Geddes started experiencing issues overnight. Any deployments using the geddes-standard-multinode storage class will experience issues accessing data. Engineers are currently investigating.</p>
]]></description>
				<pubDate>Thu, 29 Jan 2026 08:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Gilbreth Scheduling paused due to Scratch filesystem issue]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7597</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7597</guid>
				<description><![CDATA[<p>We’re currently addressing an issue with the Scratch filesystem on Gilbreth, and job scheduling has been temporarily paused while we work on a fix.</p>
<p>Our team is actively investigating and will post updates as soon as more information is available or 4pm EST. Thank you for your patience and understanding!</p>
]]></description>
				<pubDate>Thu, 22 Jan 2026 14:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Cluster & Data Depot Outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7596</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7596</guid>
				<description><![CDATA[<p>Data Depot and clusters began experiencing issues around 8:00AM EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.</p>
<p>We will provide an update by 2:00PM EST today.</p>
]]></description>
				<pubDate>Tue, 20 Jan 2026 08:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Weber outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7562</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7562</guid>
				<description><![CDATA[<p>The Weber cluster began experiencing issues around 8:00 AM EST. Engineers are currently diagnosing the issue and are working to identify a fix. The cluster is unreachable at this time. No ETA is currently available for return to service</p>
<p>We will provide an update by 2:00 PM EST or sooner.</p>
]]></description>
				<pubDate>Wed, 03 Dec 2025 08:00:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled gilbreth outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7560</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7560</guid>
				<description><![CDATA[<p>The gilbreth cluster began experiencing issues around 9:00 AM EST. Engineers are currently working on a solution and expect for return to service by 11:30 AM EST.</p>
]]></description>
				<pubDate>Tue, 02 Dec 2025 09:30:00 -0500</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Data Depot Outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7443</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7443</guid>
				<description><![CDATA[<p>The Data Depot storage system began experiencing issues starting around 4:30pm EDT today. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.</p>
<p>We will provide an update by 9pm.</p>
]]></description>
				<pubDate>Sat, 01 Nov 2025 16:30:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Geddes Storage Issues]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7439</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7439</guid>
				<description><![CDATA[<p>The Geddes cluster began experiencing issues with its storage system around 4 AM this morning. Some users may be experiencing <code>no space left on device</code> errors or other storage related errors on their deployments. Engineers are currently diagnosing the issue and are working to identify a fix.</p>
<p>We will provide an update by 11 AM.</p>
]]></description>
				<pubDate>Thu, 30 Oct 2025 08:30:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Data Depot outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7430</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7430</guid>
				<description><![CDATA[<p>Edit:</p>
<p>The Data Depot file system has returned to full service and scheduling has resumed on all clusters.</p>
<hr />
<p>The Data Depot storage system began experience issues starting around 9am EDT this morning. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.</p>
<p>We will provide an update by 12pm (noon).</p>
]]></description>
				<pubDate>Fri, 17 Oct 2025 09:00:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Data Depot outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7421</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7421</guid>
				<description><![CDATA[<p>Edit:</p>
<p>Data Depot functionality has been restored.</p>
<hr />
<p>The Data Depot file system began experiencing issues with writes around 2:30pm EDT. The data migration process currently ongoing from Data Depot 2 to Data Depot 3 ran into an unexpected problem. Engineers have identified the problem and are correcting it. Users may have seen &quot;no space left on device&quot; for approximately 30 minutes. Job scheduling has been paused while this issue is being addressed.</p>
<p>We will provide an update by 5 PM.</p>
]]></description>
				<pubDate>Wed, 15 Oct 2025 14:30:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Problem For New User/Projects On Anvil]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7302</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7302</guid>
				<description><![CDATA[<p>Anvil is experiencing a problem with new user and allocation propagation.</p>
<p>Our engineers are working on the fix, and will keep this updated.</p>
<p>The problem has been fixed at 5 pm.</p>
]]></description>
				<pubDate>Mon, 18 Aug 2025 11:00:00 -0400</pubDate>
									<category>Outages</category>
							</item>
					<item>
				<title><![CDATA[Unscheduled Gautschi outage]]></title>
				<link>https://rcac.purdue.edu/index.php/news/7271</link>
				<guid isPermaLink="true">https://rcac.purdue.edu/index.php/news/7271</guid>
				<description><![CDATA[<p>The Gautschi cluster began experiencing issues with cooling around 2:00pm EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.</p>
<p>No ETA at this time. We will update as soon as possible or by 5:00pm EST.</p>
]]></description>
				<pubDate>Thu, 31 Jul 2025 14:00:00 -0400</pubDate>
									<category>Outages</category>
							</item>
			</channel>
</rss>