Best practices agent skill for Red Hat Enterprise Linux
Updated
Status: Developer Preview
Note: The RHEL best practices skill is currently available as a Developer Preview. For details on the scope of support, please review the This content is not included.Red Hat Developer Preview - Scope of Support.
Overview
The best practices skill for RHEL empowers your AI tool with Red Hat expertise for troubleshooting, system health monitoring, and proactive maintenance on Red Hat Enterprise Linux (RHEL).
By providing a structured framework for diagnosing complex RHEL issues, this skill helps AI-generated advice align with Red Hat's recommendations. When combined with the MCP server for RHEL, your AI tool can read your system's state to provide contextual, data-backed guidance rather than generic advice.
Key capabilities:
- SELinux: Assisting with diagnosing "Permission denied" errors and AVC denials.
- Kernel Operations: kdump configuration, crash analysis (vmcore), and kernel live patching (kpatch).
- System Performance: Proactive health checks and Performance Co-Pilot (PCP) monitoring.
- Security & Compliance: Managing Crypto policies, FIPS mode, and compliance scanning with OpenSCAP.
- Support Readiness: Generating
sos reportsfor Red Hat Support. - Subscription Management: Helping resolve common subscription management and repository availability issues.
What is an agent skill (SKILL.md)?
This integration is built using the Agent Skills open standard (SKILL.md). Originally created by Anthropic and released as an open standard, the Agent Skills standard provides a method to equip AI tools with specific workflows, best practices, and domain knowledge.
Because Agent Skills is an open standard, you can use the best practices skill for RHEL across many modern AI client or agent tools that support the SKILL.md format.
For more details on the standard, see Content from agentskills.io is not included.agentskills.io.
Example use cases
- "Something is wrong with my RHEL system but I'm not sure what"
- "Permission denied — I think it's SELinux"
- "My RHEL server crashed / kernel panic"
- "dnf says my system is not registered"
- "I need to collect a sosreport for a Red Hat support case"
Prerequisites
To get the most out of the RHEL best practices skill, we recommend the following:
- An AI client or agent that supports
SKILL.mdintegration (goose, Cursor, Claude Code, etc.). - (Recommended) The MCP server for RHEL: While the skill provides knowledge, the Model Context Protocol (MCP) server for RHEL allows the AI to gather real time diagnostic information (like
list_servicesorget_journal_logs) from RHEL hosts. - (Optional) Integration with the MCP servers for Content from github.com is not included.Red Hat Lightspeed, Red Hat Satellite, or Red Hat Ansible Automation Platform for fleet-wide visibility and automated remediation.
How to install the skill
Because skills are designed to be client-agnostic, the installation process depends on the AI client or agent tool you are using.
General Installation Steps:
- Download or copy the RHEL best practices
SKILL.mdfile. - Consult your specific AI client's documentation for instructions on how to import, load, or reference a
SKILL.mdfile. - (Optional) If your client supports MCP servers, it is recommended to configure the MCP server for RHEL within your client's settings to enable live system diagnostics.
For exact step-by-step installation instructions, see the official documentation of your chosen AI agent tool.
How to use the skill
How you interact with the skill depends on the specific AI client or agent tool you are using:
- Automatic invocation: Many tools will automatically load and apply the skill if they determine that the skill can help with your query (for example, if you ask a question about RHEL administration, SELinux, or system troubleshooting).
- Manual invocation: Many tools also support explicitly or manually invoking a skill, such as using an
@mention, a slash command, or selecting it from a menu.
Please refer to your specific client/tool documentation for more details on how it handles skill invocation.
How to get the SKILL.md file
You can get the skill using either of the following methods:
Option 1: Download the file
Download the attached .zip file at the bottom of this page.
Option 2: Copy and paste
Copy the raw text of the skill, then paste it into a file named SKILL.md within a directory named rhel-best-practices.
---
name: rhel-best-practices
description: Surfaces RHEL best practices and system insights for troubleshooting, system health, and proactive maintenance. Use when users ask about SELinux denials (permission denied, AVC), kernel crashes or panics (kdump, vmcore, crashkernel), kernel live patching (kpatch), application crashes (coredump), failed services, subscription or registration, system performance, PCP monitoring, sosreport, FIPS or crypto policies, system hardening or compliance scanning (OpenSCAP, CIS, STIG), Red Hat Lightspeed (Insights), Satellite, Ansible Automation Platform, or patching. Also use for general "my RHEL system isn't working" questions or any RHEL troubleshooting where the user hasn't identified the root cause, even if they don't mention a specific tool or domain.
license: Apache-2.0
metadata:
author: Red Hat
version: "0.1.0"
---
# RHEL Best Practices
Surface Red Hat Enterprise Linux (RHEL) best practices and system insights during troubleshooting and proactive maintenance, using the **MCP server for RHEL** to ground every assessment in the user's actual system state. This skill identifies what's wrong and what Red Hat recommends — it provides diagnostic workflows and best-practice knowledge for common RHEL administration areas: SELinux, kdump, PCP monitoring, sosreport, FIPS/crypto compliance, OpenSCAP hardening, and health checks. It can also integrate with the MCP servers for Red Hat Lightspeed, Satellite, and Ansible Automation Platform when they are already connected (see Red Hat Ecosystem Integration), but those are optional.
## Workflow
### Step 1: Assess the system
The RHEL system being diagnosed is almost always a remote host — not the local machine. Before running any diagnostics, determine which host to target:
- If the user has already named or described a specific host, use that.
- If the user's question implies a specific system but hasn't provided a hostname, ask once: "Which host should I look at?" Do not default to running tools against the local system.
- Only target the local system if the user explicitly says so (e.g., "this machine", "localhost", "the system I'm on").
If the **MCP server for RHEL** is connected, use it as the primary diagnostic tool — it replaces the need for manual shell commands during initial assessment. **For remote systems, pass the `host` parameter** so tools run on the target RHEL host via SSH (every MCP server for RHEL tool accepts this optional argument). Omit `host` only when the user has explicitly indicated the local system is the target.
Detection: Identify the MCP server for RHEL by its tools — `get_system_information`, `get_journal_logs`, `list_services`, `get_disk_usage`, and similar RHEL system diagnostic tools. The server name in the client config is user-chosen and may vary.
| What to check | MCP tool |
-|-
| RHEL version | `get_system_information` |
| Failed services | `list_services` |
| Recent errors | `get_journal_logs` |
| CPU pressure | `get_cpu_information` |
| Memory pressure | `get_memory_information` |
| Disk space | `get_disk_usage` |
| Network state | `get_network_interfaces`, `get_network_connections` |
Use MCP data naturally: "I can see your system is running RHEL 9.3 and the httpd service has failed..." rather than asking the user to run diagnostic commands they may already know.
**If MCP is not available**, fall back to standard diagnostic commands. Use `cat /etc/redhat-release` for RHEL version (not `lsb_release`, which is not installed by default). Many best-practice domains are version-sensitive (e.g., crashkernel, FIPS, container runtime). Infer the version from contextual clues — `yum`, `authconfig`, `ntpd`, or Python 2 references suggest RHEL 7; `dnf module` or standalone `vdo` management suggests RHEL 8; `dnf5`, `bootc`, or `pasta` suggest RHEL 10. If the version is still unknown and the answer materially differs, ask the user once. Otherwise, default to RHEL 9.
### Step 2: Apply the relevant best-practice domain
Match the user's problem to the appropriate domain:
| Symptom / question | Domain |
-|-
| "Permission denied", app won't start, AVC | **SELinux Troubleshooting** |
| Kernel panic, system crash, vmcore | **kdump Configuration and Crash Analysis** |
| Apply kernel security patch without rebooting | **Kernel Live Patching with kpatch** |
| "Something is wrong and I don't know what" | **Basic Troubleshooting Framework** |
| Can't register, subscription expired, repos unavailable | **Subscription and Registration Troubleshooting** |
| Need to gather data for Red Hat Support | **sosreport** |
| Routine check, "is my system healthy?" | **Proactive Health Checks** |
| Slow system, resource monitoring, baselines | **Performance Monitoring with PCP** |
| TLS/SSH policy, FIPS compliance | **Crypto Policies and FIPS Compliance** |
| CIS, STIG, compliance scanning | **Compliance Scanning with OpenSCAP** |
Then follow that domain's reference workflow.
### Step 3: Provide a clear path forward
End every interaction with a diagnosis and a recommended path, not just raw commands:
- What is wrong and why (grounded in system data when MCP is available)
- What Red Hat recommends (the best practice for this situation)
- How to remediate (see Remediation Approach)
- If the issue requires Red Hat Support: how to generate a sosreport and open a case
Before presenting any command that makes a persistent system change (kernel parameters, SELinux policy, crypto settings, firewall rules with `--permanent`), explain what it does and whether it is reversible. If an undo command exists, provide it alongside the original.
## Remediation Approach
This skill is an advisor — it diagnoses problems and recommends solutions. It does not direct the AI agent to make ad-hoc system changes.
**Diagnostic actions** — commands that only read system state (e.g., `getenforce`, `ausearch`, `journalctl`, `systemctl status`, `chronyc tracking`, `kpatch list`) — are always appropriate. Use the MCP server for RHEL's diagnostic tools or recommend these commands directly.
**System modifications** — commands that change configuration, enable services, set kernel parameters, modify SELinux policy, or alter firewall rules — should be presented as *what needs to change*, not as steps for the AI agent to execute. Do not proactively execute system-modifying commands through MCP tools, shell access, or any other path unless the user explicitly requests it.
Red Hat recommends deterministic automation for RHEL system modifications — automation provides consistent, repeatable outcomes every time it runs. Every RHEL system has access to `ansible-core` and the `rhel-system-roles` package for foundational system configurations. Surface this when contextually relevant (the user asks about automation, repeatability, or scaling a fix). Do not push it on every interaction or on users who are not using automation tooling.
**When presenting remediation:**
1. Explain what needs to change and why
2. Provide the specific commands as reference material — they document what the fix involves and serve users who choose to remediate manually or through their own tooling
## Response Guidelines
**Adapt to the user.** Read expertise and urgency from their language. A sysadmin who says "my production server just crashed" needs immediate triage — lead with the most likely fix, not a systematic assessment. A user asking "is my system healthy?" is in exploration mode and benefits from a thorough walkthrough. An experienced admin asking a specific question wants a concise answer.
**Red Hat ecosystem enrichment.** If the **MCP server for Red Hat Lightspeed** or the **MCP server for Satellite** happens to be connected, use them to enrich the answer with fleet-wide or proactive data — see the Red Hat Ecosystem Integration section. If they are not connected, do not prompt the user to install or register.
**Linked references.** Links to Red Hat resources in this skill may require authentication. If linked content is inaccessible, present the link to the user rather than omitting or paraphrasing the reference.
---
## Best-Practice Domains
The workflows below document each domain's diagnostic steps and remediation procedures. Diagnostic commands can be recommended directly. Remediation commands are reference material showing what changes are needed — see Remediation Approach for how to present these to the user.
### SELinux Troubleshooting
SELinux is RHEL's mandatory access control system. It is enabled and set to enforcing by default. **Never recommend disabling SELinux.** Instead, fix the specific denial.
**Checking SELinux status:**
bash
getenforce
sestatus
**When a user reports a "Permission denied" or an application that won't start:**
1. Check for AVC denials:
bash
sudo ausearch -m AVC,USER_AVC,SELINUX_ERR -ts recent
2. Understand the denial:
bash
sudo ausearch -m AVC,USER_AVC,SELINUX_ERR -ts recent | audit2why
3. Apply the targeted fix based on what `audit2why` reports:
**If the file context is wrong** (most common):
bash
sudo restorecon -Rv /path/to/affected/directory
**If an SELinux boolean needs to be enabled:**
bash
sudo semanage boolean -m --on <boolean_name>
sudo setsebool -P <boolean_name> on
**If a non-standard port needs labeling:**
bash
sudo semanage port -l | grep <service_type>
sudo semanage port -a -t <service_type>_port_t -p tcp
**If a custom policy module is needed** (last resort):
bash
sudo ausearch -m AVC,USER_AVC,SELINUX_ERR -ts recent | audit2allow -M mypolicy
sudo semodule -i mypolicy.pp
**Common booleans that users frequently need:**
bash
sudo setsebool -P httpd_can_network_connect on
sudo setsebool -P httpd_enable_homedirs on
sudo setsebool -P samba_enable_home_dirs on
sudo semanage boolean -m --on httpd_can_sendmail
**Reverting changes:** If a fix was wrong or too broad, undo it before trying the next approach:
- Boolean: `sudo setsebool -P <boolean_name> off`
- Port label: `sudo semanage port -d -t <service_type>_port_t -p tcp <port>`
- Custom module: `sudo semodule -r mypolicy`
- Note: `restorecon` resets to default contexts and cannot be undone if a custom context was intentionally set. Use `semanage fcontext -l` to review custom rules before running it broadly.
**Key principle:** Work through denials one at a time. Fix, test, repeat. Avoid `setenforce 0` as a long-term solution.
**If no denials appear in audit.log (dontaudit rules):**
When an application is blocked but nothing shows in `/var/log/audit/audit.log`, the denial is likely silenced by a **dontaudit** rule. These rules suppress logging for common, harmless probes to prevent log bloat.
1. Disable dontaudit rules to expose hidden denials:
bash
sudo semodule -DB
(`-D` disables dontaudit rules, `-B` rebuilds and reloads the policy.)
2. Reproduce the issue, then search for the now-visible denials:
bash
sudo ausearch -m AVC -ts recent
sudo ausearch -m AVC -ts recent | audit2why
3. Re-enable dontaudit rules immediately after — leaving them disabled causes excessive log growth:
bash
sudo semodule -B
**Inspect existing dontaudit rules without changing policy** (requires `setools-console`):
bash
sesearch --dontaudit
sesearch --dontaudit -s httpd_t # filter by source domain
### kdump Configuration and Crash Analysis
kdump captures the contents of system memory when a kernel crash occurs. This vmcore dump is critical for diagnosing the root cause of kernel panics.
**Checking kdump status:**
bash
systemctl status kdump
If kdump is not active, enable it:
bash
sudo systemctl enable --now kdump
**Verifying the crashkernel parameter:**
bash
cat /proc/cmdline | grep crashkernel
Crashkernel reservation differs by version:
- **RHEL 7–8:** `crashkernel=auto` is the standard default. If missing, add it:
`sudo grubby --update-kernel=ALL --args="crashkernel=auto"` (reboot required).
To revert: `sudo grubby --update-kernel=ALL --remove-args="crashkernel=auto"` (reboot required).
- **RHEL 9+:** `crashkernel=auto` is **not supported**. The `kexec-tools` package handles reservation automatically. To query the default value: `kdumpctl get-default-crashkernel`. If a manual override is needed, specify an explicit size (e.g., `crashkernel=256M`), not `auto`.
To revert a manual override: `sudo grubby --update-kernel=ALL --remove-args="crashkernel=256M"` (reboot required).
**Configuration file:** `/etc/kdump.conf`
Key settings:
- `path /var/crash` — where vmcores are saved (default)
- `core_collector makedumpfile -l --message-level 1 -d 31` — compression and filtering
- `default reboot` — action after dump completes
**When a crash occurs:**
1. After reboot, check for a vmcore:
bash
ls -la /var/crash/
2. For initial analysis, use the `crash` utility:
bash
sudo dnf install crash kernel-debuginfo
sudo crash /usr/lib/debug/lib/modules/$(uname -r)/vmlinux /var/crash/
3. Inside `crash`, common commands: `bt` (backtrace), `log` (kernel log), `ps` (process list at time of crash), `sys` (system info).
4. If filing a Red Hat Support case, attach the vmcore or the output of `crash` analysis.
**Red Hat's Kdump Helper** ([https://access.redhat.com/labs/kdumphelper/](https://access.redhat.com/labs/kdumphelper/)) can assist with configuration.
**Application-level crash dumps:** For userspace process crashes (not kernel panics), RHEL 9+ uses `systemd-coredump` by default — core dumps are stored in the journal and managed via `coredumpctl`. Use `coredumpctl list` to see recent crashes and `coredumpctl info <PID>` for details. ABRT, which handled this on RHEL 7–8, was removed in RHEL 9.
### Kernel Live Patching with kpatch
kpatch applies critical kernel security patches to a running kernel without requiring a reboot — useful for systems where unplanned downtime is unacceptable.
Red Hat supports live patching on a specific subset of kernels called kpatch-eligible base errata kernels. For a current list of supported kernels with GA/EOL dates and CVEs addressed, see [Kernel Live Patch life cycles](https://access.redhat.com/articles/4499631).
**Cadence and Eligibility:**
- Release Cycle: Quarterly (roughly every 3 months).
- Support Window: Each quarterly kernel receives live patches for Critical and Important CVEs for up to one year from its release. Upgrading to the next quarterly kernel resets the reboot clock — you get another year of live patch coverage before a reboot is required.
**Support Life Cycles by RHEL Release:**
| Release Type | Live Patch Eligibility |
-|-
| Enhanced EUS / E4S | 4 years |
| Non-EUS (standard minor release) | 6 months |
When a kernel's support window ends, update to a current kpatch-eligible kernel to continue receiving live patches.
**Example Security SLA Strategy:**
A typical enterprise strategy using the quarterly cadence to balance security with uptime:
- Critical CVEs (SLA: 48h): Apply via live patch (no reboot).
- Important CVEs (SLA: 2 weeks): Apply via live patch (no reboot).
- Moderate and below: Patch by updating to the latest kpatch-eligible kernel and rebooting every 3-6 months.
**Setup and Automation (DNF Filtering Plugin):**
To prevent accidentally installing a kernel that does not support live patching, use the kpatch DNF plugin. This filters the errata list to show only kpatch-eligible kernels.
bash
sudo dnf install kpatch-dnf
sudo dnf kpatch auto
This installs the `kpatch-dnf` DNF plugin and subscribes all currently installed kernels — and any future kernels — to receive live patches automatically. When new patches are released, `dnf update` applies them.
**Verify loaded patches:**
bash
kpatch list
**DNF plugin options:** The commands `auto` and `manual` are equivalent to `auto-update` and `manual-update`.
| Option | Behavior |
-|-
| `sudo dnf kpatch auto-update` | Subscribe all currently installed kernels — and any future kernels — to receive live patches automatically |
| `sudo dnf kpatch manual-update` | Switch to manual mode — existing patches stay loaded, but new kernels will not automatically receive live patches |
| `sudo dnf kpatch auto-filter` | Hide kpatch-unsupported kernels from DNF transactions — only kernels with a corresponding `kpatch-patch-<kernel-version>` package are allowed into the transaction |
| `sudo dnf kpatch no-filter` | **(default)** Do not filter — kernels are updated normally regardless of live-patch availability |
Red Hat's z-stream kernel release cadence increased, and live-patch coverage was reduced to a subset of kernels on a quarterly basis. Without `auto-filter`, a routine `dnf update` can upgrade the running kernel to a version that has no live patch available, silently breaking kpatch coverage. Recommend `auto-filter` on any system where continuous live-patch coverage is a requirement — for example, systems that must remain unrebooted between maintenance windows or that are subject to strict compliance requirements.
**Limitations:** Not all CVEs can be addressed through live patching — data structure changes and complex fixes may still require a traditional kernel update and reboot. Live patches are available for the latest minor release kernel; coverage ends when the next minor release ships (unless using Extended Update Support). You must not use SystemTap or kprobe tools while a patch is loaded — the kpatch might not take effect until the probes are removed.
**When to recommend kpatch vs a reboot:** kpatch is a bridge, not a replacement. It buys time between maintenance windows. Systems should still plan regular kernel updates with reboots. If `dnf needs-restarting -r` indicates a kernel update is pending and a maintenance window is available, a reboot is the more thorough approach.
### Basic Troubleshooting Framework
A systematic approach to "something is wrong and I don't know what." The agent already knows standard Linux diagnostic commands — the value here is the triage order and the RHEL-specific steps.
**Triage order:**
1. Failed services and recent error logs (use MCP `list_services` / `get_journal_logs`, or standard systemctl/journalctl)
2. Resource pressure — CPU, memory, disk space, inodes (use MCP tools, or standard top/free/df)
3. Network connectivity (use MCP `get_network_interfaces` / `get_listening_ports`, or standard ip/ss)
4. **Time synchronization** — `chronyc tracking` (clock skew causes authentication failures, TLS errors, log correlation issues, and Kerberos ticket rejection)
5. **SELinux denials** — `ausearch -m AVC -ts recent` (frequently the root cause on RHEL; see SELinux Troubleshooting domain)
6. **Subscription and update status** — `subscription-manager status` and `dnf check-update` (RHEL-specific; expired subscriptions block updates)
### Subscription and Registration Troubleshooting
RHEL requires registration with Red Hat to access repositories and updates. All Red Hat accounts now use **Simple Content Access (SCA)**, which grants access to all content in the subscription portfolio without manually attaching subscriptions to individual systems — register and enable repos is all that's needed.
**Quick diagnostics:**
bash
sudo subscription-manager identity # Is the system registered? Shows org and system UUID
sudo subscription-manager status # Overall subscription compliance
sudo subscription-manager repos --list-enabled # Which repos are currently active
**Common issues and fixes:**
**System not registered** ("`This system is not registered`" blocking `dnf`):
bash
sudo subscription-manager register --username=
Or with an activation key (preferred for automation and Satellite-managed systems):
sudo subscription-manager register --activationkey=
**Registered but repos unavailable:**
bash
sudo subscription-manager refresh # Re-download content access certificate
sudo subscription-manager repos --enable=
If `refresh` alone does not resolve the issue, try `sudo subscription-manager refresh --force`. This is typically needed after changes to `/etc/rhsm/rhsm.conf` that affect how `redhat.repo` is regenerated.
**SSL/certificate errors during registration:** Usually caused by clock skew (`timedatectl` to check, `chronyc makestep` to fix), a proxy intercepting HTTPS traffic, or corrupted RHSM certificates (`sudo dnf reinstall subscription-manager-rhsm-certificates`).
**Developer subscription specifics:** The free Red Hat Developer Subscription for Individuals auto-renews annually. If it lapses, renew it at developers.redhat.com — the system itself does not need to be re-registered; it will resume access once the subscription is active again.
**Subscription-manager log:** `/var/log/rhsm/rhsm.log` contains detailed registration and certificate activity for deeper troubleshooting.
### sosreport (System Information Collection)
`sos report` collects configuration files, logs, and system data into an archive that Red Hat Support uses to diagnose problems.
**When to generate one:**
- Before opening a Red Hat Support case
- When asked by Red Hat Support
- When documenting the state of a system before or after a change
**Generating a report:**
bash
sudo sos report
The command is interactive by default and asks which plugins to enable. The default set covers most cases. The output archive is saved to `/var/tmp/sosreport-*.tar.xz`.
**Useful options:**
bash
sudo sos report --batch # non-interactive
sudo sos report -o
sudo sos report --clean # obfuscate hostnames and IPs
sudo sos report --upload # generate the report and upload it to Red Hat
On RHEL 8+, the command is `sos report` (not `sosreport` — the package is `sos`, the command changed).
**Uploading to Red Hat Support:**
Upload the archive through the Red Hat Customer Portal (support case attachments). Alternatively, use `sos report --upload` to generate and upload in one step or `sos upload <file>` to upload a previously generated report or other file. On RHEL 7–8, the `redhat-support-tool addattachment` command was also available, but `redhat-support-tool` is deprecated in RHEL 8 and not shipped in RHEL 9+.
### Proactive Health Checks
Run these RHEL-specific checks when the user asks "is my system healthy?", when performing a routine review, or when no specific symptom has been identified but the user wants a general assessment. Standard resource checks (CPU, memory, disk, uptime) are left to the agent or the MCP server for RHEL. The items below are RHEL-specific checks the agent should include:
| What to check | Command | Why it matters |
-|-|-
| Subscription status | `subscription-manager status` | Expired subscription blocks updates |
| Security errata | `dnf updateinfo list --updates --security` | Shows applicable RHSA advisories |
| Pending updates | `dnf check-update` | Shows all available updates |
| SELinux denials (24h) | `ausearch -m AVC -ts today` | Catch silent denials before they cause outages |
| kdump readiness | `systemctl is-active kdump` | Ensures crash dumps will be captured |
| kpatch status | `kpatch list` | Shows whether kernel live patches are loaded |
| Lightspeed registration | `insights-client --status` | Confirms proactive monitoring is active |
| Crypto policy | `update-crypto-policies --show` | Verify expected policy is active |
If the MCP server for RHEL is available, resource checks (disk, memory, CPU, services) can be performed through its tools instead of shell commands.
### Performance Monitoring with PCP
Performance Co-Pilot (PCP) is RHEL's standard performance analysis toolkit — preferred over ad-hoc sysstat/sar usage for persistent, structured monitoring.
**Quick start:**
bash
sudo dnf install pcp-zeroconf
`pcp-zeroconf` sets up PCP with sensible defaults — it enables and starts `pmcd`, `pmlogger`, and `pmie` automatically, and configures log archiving to `/var/log/pcp/`.
**Real-time monitoring:**
bash
pmstat # vmstat-like overview
pmrep :sar-u # CPU utilization (sar-compatible)
pmrep disk.dev.read disk.dev.write # disk I/O
pminfo -f mem.util.available # specific metric
**Reviewing historical data:**
bash
pmrep -a /var/log/pcp/pmlogger/$(hostname)/<YYYYMMDD.HH.MM> :sar-u
pmdumplog -l /var/log/pcp/pmlogger/$(hostname)/<YYYYMMDD.HH.MM>
**Grafana integration** (for dashboards):
bash
sudo dnf install grafana grafana-pcp
sudo systemctl enable --now grafana-server pmproxy
Access Grafana at `http://<hostname>:3000`. Add the PCP Redis data source.
PCP replaces the need for standalone monitoring agents. It collects hundreds of kernel, process, disk, network, and application metrics out of the box, with extensible PMDAs (Performance Metric Domain Agents) for databases, containers, and custom metrics.
### Crypto Policies and FIPS Compliance
RHEL uses system-wide crypto policies to enforce consistent cryptographic settings across all applications (TLS, SSH, IPsec, Kerberos).
**Check current policy:**
bash
update-crypto-policies --show
**Available policies:** DEFAULT, LEGACY (weaker, for backward compat), FUTURE (stricter, forward-looking), FIPS.
**RHEL 10 policy changes:** DEFAULT is stricter than on earlier versions — it rejects RSA key exchange in TLS. LEGACY no longer allows SHA-1 HMAC in TLS.
**Set a policy:**
bash
sudo update-crypto-policies --set FUTURE
Reboot after changing crypto policies to ensure all services pick up the new settings. A stricter policy may break connections to legacy clients or older systems. To revert, re-apply the previous policy (e.g., `sudo update-crypto-policies --set DEFAULT` if DEFAULT was the prior setting) and reboot again.
**Enabling FIPS mode:**
On RHEL 8–9:
bash
sudo fips-mode-setup --enable
sudo reboot
fips-mode-setup --check
On RHEL 10: `fips-mode-setup` is removed. FIPS must be enabled at install time by adding `fips=1` to the kernel command line. To verify:
bash
cat /proc/sys/crypto/fips_enabled # returns 1 if active
### Compliance Scanning with OpenSCAP
OpenSCAP automates security compliance scanning against industry benchmarks (CIS, DISA-STIG, ANSSI, PCI-DSS, HIPAA, and others).
**Install:**
bash
sudo dnf install openscap-scanner scap-security-guide
**Discover available profiles:**
bash
oscap info /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml
Data stream files are shipped by `scap-security-guide` in `/usr/share/xml/scap/ssg/content/`. Replace `ssg-rhel9-ds.xml` in these examples with the file matching the target RHEL version (`ssg-rhel8-ds.xml`, `ssg-rhel9-ds.xml`, or `ssg-rhel10-ds.xml`). For a full overview of available compliance policies, see [Red Hat Product Compliance](https://access.redhat.com/compliance).
**Scan against a profile:**
bash
sudo oscap xccdf eval
--profile xccdf_org.ssgproject.content_profile_cis
--results-arf results.xml
/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml
**Generate HTML report:**
bash
oscap xccdf generate report results.xml > report.html
**Tailoring profiles** (customize which rules apply):
bash
autotailor -o tailoring.xml
--select rule_id --unselect rule_id
/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml xccdf_profile_id
Compliance profiles can be applied through Image Builder, Kickstart, or Image Mode RHEL (bootc, RHEL 10+) for systems hardened from first boot.
RHEL 10 changes: `oscap-anaconda-addon` removed (no Security Policy spoke in installer). SCAP Workbench GUI removed; use `oscap` + `autotailor` CLI or the [Compliance service](https://console.redhat.com/insights/compliance) for full GUI-based tailoring.
If the **MCP server for Red Hat Lightspeed** is connected, the Compliance service can track OpenSCAP policy adherence across registered systems.
---
## Red Hat Ecosystem Integration
### Red Hat Lightspeed (formerly Insights)
Red Hat Lightspeed (formerly Red Hat Insights) is a proactive monitoring and remediation service included with every RHEL subscription. It analyzes registered systems and provides recommendations for security, performance, availability, and stability. Users may still refer to it as "Insights."
**Detecting the Insights client on the system (via the MCP server for RHEL):**
Use `list_files` to check for `/etc/insights-client/insights-client.conf`, or use `get_service_status` to check the `insights-client` timer. As with all diagnostics, pass the `host` parameter to target the remote RHEL system.
**If the system is registered:**
Mention that the user can view Advisor recommendations, vulnerability reports, compliance data, and drift analysis on the Hybrid Cloud Console (console.redhat.com). For the current issue, note any relevant capabilities:
- SELinux issues → Advisor may have specific recommendations
- Vulnerability concerns → Insights Vulnerability service shows CVE exposure
- Compliance → Insights Compliance tracks OpenSCAP policy adherence
- Patching → Insights Patch service shows applicable errata
**If the client is not installed or not registered:** Only mention the installation steps if the user asks about Lightspeed/Insights specifically, or if the current problem would clearly benefit from its data (e.g., vulnerability assessment, compliance tracking). Do not suggest installation unprompted. If appropriate:
bash
sudo dnf install insights-client
sudo insights-client --register
**MCP server for Red Hat Lightspeed:**
If the MCP server for Red Hat Lightspeed is connected, discover its available tools and use them to enrich troubleshooting. Relevant capabilities include proactive recommendations (Advisor), CVE exposure analysis, remediation playbook generation, system inventory, and upgrade planning.
Detection: Check if the MCP server for Red Hat Lightspeed (identifiable by Lightspeed service tooling — advisor, inventory, vulnerability, and remediation tools) is available among connected MCP servers.
### Red Hat Satellite
Red Hat Satellite provides fleet-level management for RHEL systems: content management, patching, provisioning, and reporting.
**If the MCP server for Satellite is connected**, discover its available tools and use them to complement per-host troubleshooting with fleet-wide visibility — errata status, patching compliance, host inventory, and infrastructure reporting. The MCP server for Satellite operates in read-only mode.
**When to prefer the MCP server for Satellite over the MCP server for RHEL:**
- Fleet-wide questions: "Which hosts need patching?" "How many systems are behind on updates?"
- Group-level concerns: "What's the patching status of my production hosts?"
- Infrastructure overview: "Show me all subnets and their host counts."
**When to prefer the MCP server for RHEL:**
- Single-host troubleshooting: "Why is this service failing?" "What's using all the disk space?"
- Real-time diagnostics: CPU, memory, processes, logs on a specific system.
Detection: Check if the MCP server for Satellite (identifiable by Satellite or Foreman tooling) is available among connected MCP servers.
Reference: Search Red Hat docs for "Connecting AI applications to the MCP server for Satellite" under the latest Satellite version.
### Red Hat Ansible Automation Platform
For enterprise-grade automation and governance on RHEL, Red Hat Ansible Automation Platform (AAP) is the recommended solution. AAP provides deterministic execution of pre-tested automation, RBAC, approval workflows, full audit trails, and centralized management at scale. Its MCP server (available as a Technology Preview in AAP 2.6+) provides a governance-first interface for AI agents — agents operate within the authenticated user's RBAC boundaries, ensuring that automation discovery and execution are governed by the same enterprise controls as human-initiated operations.
The general principle of recommending deterministic automation for system modifications is covered in the Remediation Approach section above. This section covers specifically what to do when the MCP server for Ansible Automation Platform is connected.
**If the MCP server for Ansible Automation Platform is connected**, discover its available tools and use them to offer automation-based remediation for issues identified during troubleshooting — launching job templates, querying inventory, and reviewing job logs. When offering remediation, check whether a relevant job template exists in AAP. If one does, offer it as the primary remediation path — AAP enforces RBAC and provides an audit trail.
**When to provide commands directly instead of AAP remediation:**
- One-off diagnostic commands on a single host (read-only — always appropriate)
- The user is explicitly asking for the command syntax, not an automation workflow
**If the MCP server for Ansible Automation Platform is not connected**, do not prompt the user to install or configure AAP. If the user asks about Ansible, automation, or applying a fix across systems, mention that every RHEL system includes `ansible-core` and the `rhel-system-roles` package (`dnf install rhel-system-roles`) for foundational system configuration via Ansible. Provide remediation commands as reference material for users who choose to remediate manually or through their own tooling.
Detection: Check if the MCP server for Ansible Automation Platform (identifiable by AAP tooling — tools that reference job templates, inventories, and automation workflows) is available among connected MCP servers. The MCP server for Ansible Automation Platform may operate in read-only mode (queries only) or read-write mode (can launch jobs); respect whichever mode is configured.
Reference: See "Deploying Ansible MCP server" in the Ansible Automation Platform 2.6 documentation.
---
## Terminology Notes
- **MCP server for RHEL** — the marketing-approved product name. The upstream package and binary are `linux-mcp-server`.
- **sosreport** vs **sos report** — the package is `sos`. On RHEL 8+, the command is `sos report` (two words). Older documentation may reference `sosreport` as one word.
- **vmcore** — the memory dump file produced by kdump. Saved to `/var/crash/` by default.
- **Hybrid Cloud Console** — the correct name for console.redhat.com. Not "Red Hat Console" or "Cloud Console."
- **Red Hat Lightspeed** — the current name for the proactive monitoring and AI service platform, formerly Red Hat Insights. The client package is still `insights-client`. The MCP server is the **MCP server for Red Hat Lightspeed** (container image: `red-hat-lightspeed-mcp`). Users may still refer to the service as "Insights."
- **Satellite** — "Red Hat Satellite" in full. The **MCP server for Satellite** connects to Satellite inventory via API.
- **Ansible Automation Platform (AAP)** — "Red Hat Ansible Automation Platform" in full. The **MCP server for Ansible Automation Platform** connects to AAP via API. Technology Preview in AAP 2.6+.
- **crashkernel** — the kernel boot parameter that reserves memory for kdump. One word, lowercase.
- **AVC** — Access Vector Cache. The log entry type for SELinux denials.
- **RHEL web console** — preferred over "Cockpit" in user-facing text. "Cockpit" is the upstream project name and appears in package/service names, but Red Hat docs use "the web console."
Product(s)
Category
Article Type