Infrastructure News Type: Outage Full

Affected Infrastructure: anvil.purdue.access-ci.org, anvil-gpu.purdue.access-ci.org

Start Date: June 14, 2025, 4:30 p.m.

End Date: June 14, 2025, 7:40 p.m.

Update 3:40PM, EDT June 14, 2025 Our engineers have brought Anvil back to full service. If you have any questions about this outage, please submit a ticket through ACCESS Help Desk at https://support.access-ci.org/help-ticket. Thank you. Original Post: Shortly after 12:30pm June 14, 2025, we had a major power outage at our data center at Purdue. Anvil has been impacted and will be offline until power is resumed. Our engineering team is working closely with campus power engineers to bring the power and Anvil back. We apologize for any inconvenience it might have caused. There is no ETA yet, but we will provide an update as soon as we have one. Please submit a ticket through ACCESS Help Desk at https://support.access-ci.org/help-ticket if you have any questions.

Posted: March 20, 2026

Infrastructure News Type: Reconfiguration

Affected Infrastructure: registry.access-ci.org

Start Date: June 11, 2025, 1:00 p.m.

End Date: June 11, 2025, 2:00 p.m.

On June 11, 2025, the LDAP servers previously used by the ACCESS User Registry (https://registry.access-ci.org/) will be decommissioned. No downtime is expected. (For previous related news, see https://operations.access-ci.org/node/842 .) On April 9, 2025, the ACCESS SSH Pubkey Downloader was reconfigured to use DynamoDB instead of LDAP. On April 24, 2025, the LDAP provisioner was removed from the ACCESS User Registry (https://registry.access-ci.org/) in favor of storing user attributes in DynamoDB (https://aws.amazon.com/dynamodb/). Thus, the ACCESS LDAP servers are no longer necessary. The LDAP servers for the DEV (https://registry-dev.access-ci.org/ (https://registry-dev.access-ci.org/))) and TEST (https://registry-test.access-ci.org/ (https://registry-test.access-ci.org/))) user registries have already been decommissioned. This final step will decommission the LDAP servers for the PROD user registry (https://registry.access-ci.org/ (https://registry.access-ci.org/))).

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: bridges2-em.psc.access-ci.org, bridges2-gpu.psc.access-ci.org, bridges2-rm.psc.access-ci.org, bridges2-ocean.psc.access-ci.org

Start Date: June 11, 2025, 5:01 a.m.

End Date: June 11, 2025, 5:05 p.m.

Bridges-2 and Neocortex have returned to service shortly after noon Eastern time. =============================================================== At around Midnight Eastern time, the Pittsburgh Supercomputing Center machine room experienced a power disruption which caused many of the systems to fail. Our team is working to restore all systems to full capacity. Jobs which failed due to the incident will be automatically refunded. Thank you for your patience while we work through this issue. If you have any specific quesions or problems, please contact us by sending email to help@psc.edu

Posted: March 20, 2026

Infrastructure News Type: Outage Partial

Affected Infrastructure: anvil.purdue.access-ci.org, anvil-gpu.purdue.access-ci.org

Start Date: June 10, 2025, 2:00 p.m.

End Date: June 10, 2025, 11:30 p.m.

Starting at 10:00 AM EST, Anvil began experiencing a network interruption. This may impact access to services such as Open OnDemand and Remote Desktop. Additional impacts are still being assessed. At this time, we do not have an estimated time for service restoration. We will provide updates as more information becomes available. Thank you for your patience and understanding. Updates: The issue was resolved at 7:30 PM EST on June 10. All services should now be functioning normally. If you still experience any problem, please feel free to reach out Anvil support team through ACCESS Help Desk (https://support.access-ci.org/help-ticket).

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: aces.tamu.access-ci.org

Start Date: June 4, 2025, 2:00 p.m.

End Date: June 5, 2025, 5:00 p.m.

The ACES cluster will be unavailable during maintenance from 9am to 8pm CDT on Wednesday June 4. A reservation is in place to prevent jobs from running past the start time of the maintenance period. The maintenance is currently extended to 12pm CDT Thursday.

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: expanse.sdsc.access-ci.org, expanse-gpu.sdsc.access-ci.org, expanse-ps.sdsc.access-ci.org

Start Date: May 31, 2025, 9:30 p.m.

End Date: June 2, 2025, 6:30 a.m.

>>>>> Update 1 Dear Expanse User, Expanse was put back in production after recovery from the power outage we had in the machine room (and UCSD wide). The machine is available for use and running jobs. Voyager has also been brought back into production. Thanks SDSC User Services Staff >>>>> There was a power outage at UCSD that impacted the SDSC machine room. The systems at SDSC (Expanse, Voyager) are currently down. We will update once they are brought up and accessible.

Posted: March 20, 2026

Infrastructure News Type: Outage Partial

Affected Infrastructure: tickets.access-ci.org

Start Date: May 29, 2025, 2:00 p.m.

End Date: June 5, 2025, 2:00 p.m.

Hi Everyone, The ACCESS Operations team is working on optimizing the ticketing system by streamlining the dropdown options available on portal forms. These improvements are intended to enhance the user experience and make ticket categorization more intuitive. We are planning to roll out these changes on June 6, and the implementation window will take place from May 29 to June 5. During this period, there may be minor outages or brief disruptions. Updated Operational Support Issues: - Allocations (includes AMIE, XRAS, etc.) - Support (includes OnDemand, Pegasus, Knowledge Base, Affinity Groups, Events, Announcements, Ask.CI, etc.) - Security and Authentication (includes IAM, policies, etc.) - Networking and Data Transfer (includes SSL, DNS, CONECTnet, etc.) - Operations Infrastructure (includes monitoring, logging, GitHub, etc. - Operations Software and Online Services (includes CiDeR, portal, dashboard, API, etc.) - Metrics - ACCESS Communications and Collaboration Tools - Ticket System - Resource Integration - Some Other Question Please note that minor updates to queues and watcher groups will also be included as part of this change. We appreciate your patience and understanding as we work to improve the system. We'll keep you informed with any further updates. Thanks & Regards, Dinuka (On Behalf of ACCESS Operations)

Posted: March 20, 2026

Infrastructure News Type: Outage Partial

Affected Infrastructure: expanse.sdsc.access-ci.org, expanse-gpu.sdsc.access-ci.org, expanse-ps.sdsc.access-ci.org

Start Date: May 25, 2025, 10:45 p.m.

End Date: May 29, 2025, 12:00 a.m.

>>> Update #3 Dear Expanse User, We have remounted the Lustre filesystem on Expanse excluding the OST that is having problems. This should prevent the hung tasks that were causing higher loads on the login nodes. We continue to work on the OST with issues and will update once it is returned to service. In the interim any old files that were striped onto the OST will fail on reads. New I/O to the filesystem will target healthy OSTs. Thanks SDSC User Services Staff >>> Update #2 Dear Expanse User, We are still working on the one OST from the Expanse Lustre filesystem that is failing to mount. This is making all files/directories that are striped onto this OST unavailable. Please note that this will also cause full file listings to hang so please refrain from doing full metadata listings on Lustre directories. All other OSTs on the filesystem are usable and new I/O will automatically avoid the problem OST. We will update once the OST with issues is restored. Thanks SDSC User Services Staff >>> Update Dear Expanse User, We brought the two OSSs online last night but there is still one storage target on one of them that needs more work to recover. We are continuing to look at the issue and will update again once the filesystem is back. Thanks SDSC User Services >>> Dear Expanse User, We are currently seeing problems with two object storage servers (OSSs) that are part of the Expanse Lustre filesystem. This is causing access issues on files that are striped on these servers. Please refrain from doing full metadata listings on Lustre directories as chances are you will access a file that is on one of the OSSs and the commands might hang. We are working on resolving the problem and will update once the OSSs are back in service. Thanks SDSC User Services Staff

Posted: March 20, 2026

Infrastructure News Type: Degraded

Affected Infrastructure: tickets.access-ci.org

Start Date: May 20, 2025, 5:30 p.m.

End Date: May 23, 2025, 1:30 p.m.

Atlassian is investigating cases of degraded performance for all Jira Work Management, Jira Service Management, Jira, and Jira Product Discovery Cloud customers. Check theirstatus page (https://jira-service-management.status.atlassian.com/incidents/nmg7dw0vwtr7) for current info.

Posted: March 20, 2026

Infrastructure News Type: Outage Partial

Affected Infrastructure: aces.tamu.access-ci.org

Start Date: May 7, 2025, 1:00 p.m.

End Date: May 7, 2025, 10:00 p.m.

The nodes and devices attached to the composability fabrics in ACES Racks 2 and 3 will be offline for maintenance on Wednesday, May 7 from 8:00 AM CDT to 5:00 PM CDT.

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: launch.tamu.access-ci.org

Start Date: May 7, 2025, 1:00 p.m.

End Date: May 7, 2025, 10:00 p.m.

The TAMU Launch cluster will be down for maintenance on Wednesday, May 7 from 8:00AM CDT to 5:00PM CDT.

Posted: March 20, 2026

Infrastructure News Type: Outage Partial

Affected Infrastructure: registry.access-ci.org

Start Date: April 30, 2025, 1:00 p.m.

End Date: April 30, 2025, 1:30 p.m.

On April 30, 2025, a plugin used by the ACCESS User Registry (https://registry.access-ci.org/) will be updated to address a bug which prevents some users from resetting their ACCESS CI passwords. Server instances will be restarted during this update which may cause in-progress registrations/logins to fail.

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: bridges2-em.psc.access-ci.org, bridges2-gpu.psc.access-ci.org, bridges2-rm.psc.access-ci.org, bridges2-ocean.psc.access-ci.org, neocortex-sdflex.psc.access-ci.org

Start Date: April 30, 2025, 1:00 a.m.

End Date: May 5, 2025, 6:00 p.m.

We are pleased to announce that the Bridges-2 system are now back online and available. Neocortex is fully available. Due to severe weather that impacted the Pittsburgh area on Tuesday, April 29th, our data center experienced significant disruptions. This weather event caused widespread issues across the region, leading to a state of emergency being declared in Allegheny County. The prolonged power outage affected our machine room at Northern Pike, including the building’s generator controls, resulting in the longest sustained outage in our nearly 40-year history. Our electrical provider has been working diligently to restore power, and we are grateful for their efforts. We appreciate your patience and understanding as we navigated this extreme weather event.

Posted: March 20, 2026

Infrastructure News Type: Reconfiguration

Affected Infrastructure: registry.access-ci.org

Start Date: April 24, 2025, 1:00 p.m.

End Date: April 24, 2025, 1:10 p.m.

On April 24, 2025, the ACCESS User Registry (https://registry.access-ci.org/) will be reconfigured to remove the LDAP Provisioner. No downtime is expected. LDAP was previously used for storing ACCESS User Registry attributes for consumption by third party services such as CILogon and the ACCESS SSH Pubkey Downloader. On September 10, 2024 (https://operations.access-ci.org/node/749), an alternate provisioner using DynamoDB was deployed. (DynamoDB (https://aws.amazon.com/dynamodb/) is an AWS-managed serverless NoSQL service with high performance and a 99.999% SLA.) CILogon has been using the DynamoDB deployment since that time. On April 9, 2025 (https://operations.access-ci.org/node/838), the ACCESS SSH Pubkey Downloader was reconfigured to also use the DynamoDB deployment. After the ACCESS User Registry has been reconfigured to remove the LDAP Provisioner, there will be no services using the ACCESS User Registry LDAP servers, so they will be decommissioned.

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: tickets.access-ci.org

Start Date: April 23, 2025, 3:00 p.m.

End Date: April 24, 2025, 7:46 p.m.

Hi Everyone, Atlassian is investigating the cause of the rule execution delay. This might delay the communication of the tickets for you. We will keep you posted when we have an update. Meanwhile, your patience and understanding are appreciated. https://jira-service-management.status.atlassian.com/incidents/4717qqxyt0nk Thanks

Posted: March 20, 2026

Infrastructure News Type: Reconfiguration

Affected Infrastructure: tickets.access-ci.org

Start Date: April 16, 2025, 2:40 p.m.

End Date: May 1, 2025, 1:00 p.m.

Atlassian is planning a roll out of an updated Jira navigation 'in the coming weeks.' There are some details here. We can turn it on now or just wait until Atlassian makes it the default. https://support.atlassian.com/jira-software-cloud/docs/what-is-the-new-navigation-in-jira/

Posted: March 20, 2026

Infrastructure News Type: Outage Partial

Affected Infrastructure: expanse.sdsc.access-ci.org, expanse-gpu.sdsc.access-ci.org

Start Date: April 9, 2025, 11:30 p.m.

End Date: April 10, 2025, 4:15 a.m.

The Expanse scheduler issue has been fixed and job submissions and queue commands are working now. Thanks SDSC User Services Staff ----- Dear Expanse User, We are currently seeing issues with the Expanse Slurm scheduler and troubleshooting the problem. At present new job submissions and Slurm commands are failing. We will update once the issue is resolved. Thanks SDSC User Services

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: registry.access-ci.org

Start Date: April 9, 2025, 2:00 p.m.

End Date: April 9, 2025, 3:00 p.m.

On April 9, 2025, starting at 9:00am CDT (10:00am EDT) ACCESS Operations will be updating the production deployment of the SSH Public Key retrieval service for ACCESS RPs. This will require a very brief downtime for the existing service, during which queries from RP registered to use the service may be interrupted. There will be no interruption to the ACCESS User Identity and OAuth Client Registry for this update.

Posted: March 20, 2026

Infrastructure News Type: Reconfiguration

Affected Infrastructure: identity.access-ci.org

Start Date: April 2, 2025, 2:30 p.m.

End Date: April 2, 2025, 2:40 p.m.

On April 2, 2025, the ACCESS Identity Provider (https://idp.access-ci.org/idp) was updated to address a vulnerability in the OpenSAML library (https://shibboleth.net/community/advisories/secadv_20250326.txt) used by the Shibboleth Identity Provider software. (Updated to use Shib IdP v5.1.4 (https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500367/ReleaseNotes#5.1.4-(March-27,-2025)).)

Posted: March 20, 2026

Infrastructure News Type: Outage Full

Affected Infrastructure: tickets.access-ci.org

Start Date: March 31, 2025, 1:56 p.m.

End Date: May 1, 2025, 1:00 p.m.

The community post (https://community.atlassian.com/forums/Jira-articles/It-s-here-Work-is-the-new-collective-term-for-all-items-you/ba-p/2954892) on this and a couple other related term changes are detailed.

Posted: March 20, 2026