Troubleshooting performance issue in RHEL

Updated

I/O and file system performance can be affected by any of the following factors:

  • Data write or read patterns
  • Data alignment with underlying geometry
  • Block size
  • File system size
  • Journal size and location
  • Recording access times
  • Ensuring data reliability
  • Pre-fetching data
  • Pre-allocating disk space
  • File fragmentation
  • Resource contention

So there are some questions we need to clarify before we proceed with the troubleshooting:

  1. Is this a Database or an Application system, other? Explain.
  2. Is this new or existing Hardware /OS / Software?
  3. Any recent changes in Hardware /OS / Software?
  4. Does this/these systems have shared SAN storage?
  5. Is this a node of a cluster? Type?
  6. Please describe the environment. Include items such as Load Balancers.
  7. Are there specific times/dates this issue occurs?
  8. During "poor" Performance testing, is the system pingable, ssh'able?
  9. How are you Seeing or Monitoring this poor Performance testing?
  10. When present, what impact does the issue have on the system? For example, programs or processes taking longer to complete? How much longer?

The following data will be required in order to troubleshoot performance issue:

  • sosreport

  • iostat logs during your time of most heavy load. Open a separate terminal window and execute the following command:

# iostat -Ntkx 1 | tee -a CASENUMBER-$(hostname)-iostat.txt

Let this iostat utility run for at least an hour if not more. Once the time has past, please stop this process and attach the resulting file to the case.

If we are getting the latency sporadic or spontaneous, we can use collectl which is a great tool to capture the system's performance.

Additional RHEL Performance Resources

SBR
Product(s)
Category
Components
Tags
Article Type