Friday, July 31, 2020
GPFS file-system is returned back to normal service.
Update on the GPFS filesystem performance: All OSCAR applications, which reside in GPFS runtime, and user home directories have been moved to a faster disk. Users should expect to see significant improvements in application startup times.
Up next - We are currently migrating the Scratch and Data volumes to faster disks and anticipate this will be completed within the next few days.
We thank you for your patience as we make these improvements. Please email firstname.lastname@example.org with any questions or concerns.
CCV User Services
What’s the issue?
- Oscar’s GPFS file system is experiencing slow I/O performance
What’s the cause? 2 things contributed to this issue:
- GPFS was recently upgraded but some components of the file system have not yet been moved to the faster disk
- Poor utilization of the Oscar cluster by a small number of users (thus impacting the file system for all users)
How will it be resolved?
- GPFS metadata has been moved to the faster disk
- The applications in GPFS runtime and the home directories are currently being moved to the faster disk
- The Data and Scratch directories will also need to move to a faster disk
- The small number of Oscar users who were impacting the file system had their jobs terminated and asked to work with CCV User Services
When will it be resolved?
- Sometime within the window of July 14th-July 15th, we expect GPFS performance to be significantly better, as the migration of GPFS runtime and the home directories will be complete
- The migration of Scratch and Data will begin the week of July 20th. More communication to follow regarding that work
The Oscar cluster began experiencing issues with slow I/O performance with the new GPFS file system. In order to help mitigate this issue, we are moving some components to a faster pool of disks.
We appreciate your patience as we troubleshoot this issue. We will send another update once it is resolved. Please email email@example.com with any questions.
CCV User Services Team