⏱️ Estimated Reading Time: 12,14 minutes
MRP Not Running, ORA-16766, DGMGRL Fix, SRL Creation, Broker State, Auto-Start Trigger, Oracle 19c
⚙️ Environment Referenced in This Article
Oracle Database: 19.18.0.0.0 Enterprise Edition •
Primary: 2-Node RAC, 4.8 TB OLTP, 2,800 TPS
Standby: Physical Standby with Active Data Guard enabled
Protection Mode: Maximum Availability (SYNC/AFFIRM) •
Broker: Data Guard Broker enabled
The monitoring alert arrived at 2:48 AM: "Standby apply lag crossing 90 minutes." I connected to DGMGRL immediately. SHOW CONFIGURATION confirmed it: the MRP process was not running on the standby. Every transaction committed on the primary for the past 90 minutes was sitting unprocessed in Standby Redo Logs, and the gap was growing by the second.
In my 15 years managing Oracle production environments, a stopped MRP process is one of the most common Data Guard incidents I have resolved. It is not complicated once you know which of the five root causes you are dealing with. The problem is that each cause has a completely different fix, and applying the wrong one wastes critical time.
This guide gives you the exact decision path, the diagnostic commands to identify your specific cause, and the precise fix for each scenario. In most cases the MRP process not running in Data Guard is resolved in under 5 minutes.
Figure 1: MRP Process Not Running in Oracle Data Guard, Root Causes, Decision Flow, Fix Commands and Verification Reference, Oracle 19c
1. What MRP Does and Why Its Absence Is Critical
The Managed Recovery Process (MRP0) is the Oracle background process on the standby database responsible for reading redo data from Standby Redo Logs and applying it to the standby datafiles. It is the engine that keeps the standby synchronised with the primary.
When MRP stops, the pipeline freezes. The RFS process may continue receiving redo from the primary and writing it to Standby Redo Logs, but nothing is being applied. The standby falls further behind with every committed transaction on the primary.
| Component | Role | When MRP Stops |
|---|
| RFS (Remote File Server) | Receives redo from primary, writes to SRLs | Continues running, redo still arrives |
| MRP0 (Managed Recovery) | Reads SRLs, applies redo to datafiles | Stopped, redo not applied, lag grows |
| Apply Lag | Time between redo arrival and apply | Increases continuously until MRP restarts |
| Data Loss Risk | Redo not yet applied to standby | Equals current apply lag if primary fails |
MRP Stopped vs Transport Lag:
When MRP is not running you will typically see apply lag growing while transport lag stays near zero. This is because RFS is still receiving redo (transport working) but nothing is being applied (MRP stopped). If both transport lag and apply lag are growing, the problem is in the network, not just MRP.
2. Immediate Diagnosis: Confirm MRP Is Not Running
Before applying any fix, run these three queries. The results tell you exactly which cause you are dealing with.
-- Query 1: Confirm MRP is not running (run on STANDBY)
-- If no rows returned for MRP0 = MRP is stopped
SELECT process,
status,
thread#,
sequence#
FROM v$managed_standby
WHERE process LIKE 'MRP%'
OR process LIKE 'PR%'
ORDER BY process;
-- Query 2: Check Standby Redo Logs (run on STANDBY)
-- No rows = SRLs missing = Cause 2
SELECT group#,
thread#,
bytes/1024/1024 AS size_mb,
archived,
status
FROM v$standby_log
ORDER BY thread#, group#;
-- Query 3: Check apply and transport lag (run on STANDBY or PRIMARY)
SELECT name,
value,
time_computed
FROM v$dataguard_stats
WHERE name IN ('transport lag', 'apply lag', 'apply finish time')
ORDER BY name;
-- DGMGRL: check broker apply state (run from any node)
-- dgmgrl /
-- SHOW DATABASE VERBOSE standby_db;
-- Look for: ApplyState = ON or OFF
How to Read Your Results:
No MRP0 row in v$managed_standby = Cause 1 or 5 (not started or crashed)
No rows in v$standby_log = Cause 2 (missing SRLs, fix first)
DGMGRL ApplyState = OFF = Cause 3 (broker controlling apply)
protection_level differs from protection_mode = Cause 4
Alert log shows MRP0 terminated = Cause 5 (post-crash)
3. Cause 1: MRP Was Manually Cancelled
Most Common The most frequent reason the MRP process is not running in Data Guard is manual cancellation. A DBA runs ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL to perform maintenance, apply an OS patch, or modify a standby parameter, and forgets to restart MRP afterwards.
-- Run on STANDBY database
-- Recommended: real-time apply (requires SRLs to exist)
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
USING CURRENT LOGFILE
DISCONNECT FROM SESSION;
-- Alternative: archive-based apply (if SRLs are missing)
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
DISCONNECT FROM SESSION;
-- Verify MRP started within 30 seconds
SELECT process,
status,
thread#,
sequence#
FROM v$managed_standby
WHERE process LIKE 'MRP%';
-- Expected: MRP0 with status = WAIT_FOR_LOG or APPLYING
In a recent 4.8 TB core banking environment I managed, MRP had been cancelled during an Oracle PSU patch cycle. The DBA completing the patch restarted the standby instance but did not restart MRP. The monitoring system detected the growing apply lag 47 minutes later. Restarting MRP with real-time apply cleared the lag to zero in 12 minutes using the default single-threaded apply. For faster recovery on larger lag, see Section 8 for parallel apply.
4. Cause 2: Missing Standby Redo Logs Prevent MRP from Starting
Silent Cause If v$standby_log returns no rows, Standby Redo Logs were never created on this standby. Without SRLs, the USING CURRENT LOGFILE option fails and MRP cannot perform real-time apply. In many configurations this prevents MRP from starting at all.
The confusing part: transport lag is zero (RFS is receiving redo fine) but apply lag grows because MRP cannot start. This pattern almost always means missing SRLs.
-- Step 1: Confirm SRLs are missing (run on STANDBY)
SELECT COUNT(*) AS srl_count FROM v$standby_log;
-- Result: 0 = SRLs missing, must create before restarting MRP
-- Step 2: Check online redo log size on PRIMARY
-- SRLs must match this size exactly
SELECT group#,
members,
bytes/1024/1024 AS size_mb,
status
FROM v$log
ORDER BY group#;
-- Step 3: Create SRLs on STANDBY
-- Rule: same size as online redo logs
-- Rule: (groups_per_thread + 1) SRL groups per thread
-- Example: single-instance primary with 3 x 500MB redo groups
-- Needs 4 SRL groups (3 + 1)
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1
GROUP 11 ('+DATA') SIZE 500M;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1
GROUP 12 ('+DATA') SIZE 500M;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1
GROUP 13 ('+DATA') SIZE 500M;
ALTER DATABASE ADD STANDBY LOGFILE THREAD 1
GROUP 14 ('+DATA') SIZE 500M;
-- For 2-node RAC primary also add THREAD 2 groups 21-24
-- Step 4: Verify SRLs created
SELECT group#,
thread#,
bytes/1024/1024 AS size_mb,
status
FROM v$standby_log
ORDER BY thread#, group#;
-- Step 5: Now restart MRP with real-time apply
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
USING CURRENT LOGFILE
DISCONNECT FROM SESSION;
-- Step 6: Confirm MRP switched to real-time mode
SELECT process, status FROM v$managed_standby WHERE process = 'MRP0';
-- Expected: WAIT_FOR_LOG (not WAIT_FOR_GAP)
Production Incident: 8-Hour Apply Gap from Missing SRLs
A newly rebuilt standby database for a 2-node RAC primary showed zero transport lag but growing apply lag from the first day of operation. Investigation found that SRLs had never been created. MRP was running in archive-based apply mode, only catching up once per hour when archive logs were shipped. By the time the team noticed, the standby was 8 hours behind the primary. Creating SRLs and restarting with real-time apply brought the standby current within 15 minutes.
5. Cause 3: Data Guard Broker Apply State Is OFF
Broker Override When Data Guard Broker is enabled, it controls the apply state of the standby. If an administrator ran DGMGRL> EDIT DATABASE standby_db SET STATE='OFFLINE' or disabled the database in broker, apply is set to OFF. Running a direct SQL restart works temporarily, but broker cancels MRP again within minutes.
This is why some DBAs report that MRP restarts but stops again almost immediately. The broker is winning the battle. The fix must go through DGMGRL.
-- Connect to Data Guard Broker (run on primary or standby server)
-- dgmgrl /
-- Step 1: Check current broker state
SHOW CONFIGURATION;
SHOW DATABASE VERBOSE standby_db;
-- Look for: ApplyState = OFF or State = OFFLINE
-- Step 2: Enable database if disabled
ENABLE DATABASE standby_db;
-- Step 3: Set apply state to ON
EDIT DATABASE standby_db SET STATE='APPLY-ON';
-- Step 4: Verify configuration is now healthy
SHOW CONFIGURATION;
-- Expected: SUCCESS with no ORA-16766
-- Step 5: Cross-check MRP on standby SQL
-- SELECT process, status FROM v$managed_standby
-- WHERE process LIKE 'MRP%';
When Broker Is Enabled, Always Fix Through DGMGRL
If dg_broker_start = TRUE in the standby spfile, the broker owns apply state management. A direct ALTER DATABASE RECOVER SQL command will start MRP, but broker will override and cancel it again at the next configuration sync cycle (typically within 1,5 minutes). Always check broker state before applying a direct SQL fix.
6. Cause 4: Protection Mode Conflict Causing MRP Abort
Silent Risk When the Data Guard configuration uses Maximum Protection mode, the primary requires confirmation from the standby for every redo write. If the standby connection becomes unstable or the network experiences sustained packet loss, Oracle may abort MRP to prevent data divergence between primary and standby.
-- Step 1: Check effective protection on PRIMARY
SELECT protection_mode,
protection_level -- actual vs configured
FROM v$database;
-- If they differ, primary silently fell back
-- Step 2: Check archive destination for errors on PRIMARY
SELECT dest_id,
status,
error,
gap_status
FROM v$archive_dest_status
WHERE target = 'STANDBY'
AND status != 'INACTIVE';
-- Step 3: Lower to Maximum Availability temporarily (dgmgrl /)
EDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;
-- Step 4: Restart MRP on STANDBY
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
USING CURRENT LOGFILE
DISCONNECT FROM SESSION;
-- Step 5: Monitor stability, then restore original mode if needed
-- EDIT CONFIGURATION SET PROTECTION MODE AS MAXPROTECTION;
7. Cause 5: MRP Did Not Auto-Start After Standby Crash
Post-Crash After a standby instance crash from OS reboot, power failure, or OOM kill, the standby database restarts but MRP does not start automatically unless broker or a startup trigger is configured. The standby opens in read-only or mount state with no apply running.
-- Step 1: Verify standby state after crash (run on STANDBY)
SELECT status,
database_role,
open_mode
FROM v$database;
-- Step 2: Check alert log for crash evidence (run on standby server)
-- adrci
-- adrci> SHOW ALERT -TAIL 200
-- Look for: ORA-03113, ORA-29701, or instance terminated messages
-- Step 3: Restart MRP immediately
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
USING CURRENT LOGFILE
DISCONNECT FROM SESSION;
-- Step 4: Create auto-start trigger to prevent recurrence
-- Run on STANDBY database as SYSDBA
CREATE OR REPLACE TRIGGER start_mrp_on_open
AFTER STARTUP ON DATABASE
DECLARE
v_role v$database.database_role%TYPE;
BEGIN
SELECT database_role
INTO v_role
FROM v$database;
IF v_role = 'PHYSICAL STANDBY' THEN
EXECUTE IMMEDIATE
'ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
USING CURRENT LOGFILE DISCONNECT FROM SESSION';
END IF;
EXCEPTION
WHEN OTHERS THEN NULL; -- Never block database startup
END;
/
-- Verify trigger created
SELECT trigger_name,
status
FROM dba_triggers
WHERE trigger_name = 'START_MRP_ON_OPEN';
Broker Is Better Than the Trigger:
If you use Data Guard Broker, you do not need the startup trigger. Enable broker on the standby with dg_broker_start = TRUE in the spfile and ensure the database state is set to APPLY-ON in DGMGRL. Broker automatically restarts MRP after any standby restart. The trigger is only necessary in environments without broker configuration.
8. Enable Parallel Apply for Faster Lag Recovery
If MRP has been stopped for a long time and the standby has accumulated significant apply lag, single-threaded MRP will be slow to catch up. Enabling parallel apply uses multiple CPU cores to apply redo simultaneously, reducing catch-up time by 3,8x depending on the workload.
-- Step 1: Cancel existing MRP if running
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
-- Confirm cancelled
SELECT process, status FROM v$managed_standby
WHERE process LIKE 'MRP%';
-- Expected: No rows
-- Step 2: Restart with parallel apply
-- Set degree to half the standby CPU count as a starting point
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
PARALLEL 8
USING CURRENT LOGFILE
DISCONNECT FROM SESSION;
-- Step 3: Verify parallel workers started
SELECT process,
status,
thread#
FROM v$managed_standby
WHERE process LIKE 'MRP%'
OR process LIKE 'PR%'
ORDER BY process;
-- Expected: MRP0 + PR00 through PR07 (8 parallel workers)
-- Step 4: Monitor lag reduction rate
SELECT name,
value,
time_computed
FROM v$dataguard_stats
WHERE name IN ('transport lag', 'apply lag')
ORDER BY name;
-- Run every 2 minutes to confirm lag is dropping
9. Verify MRP Is Running and Apply Lag Is Clearing
After applying any fix, run this complete verification sequence. Do not close the incident until all five checks pass.
-- Check 1: MRP is running (STANDBY)
SELECT process,
status,
thread#,
sequence#
FROM v$managed_standby
WHERE process LIKE 'MRP%';
-- Pass: MRP0 status = WAIT_FOR_LOG or APPLYING
-- Check 2: Apply lag dropping (STANDBY)
SELECT name,
value
FROM v$dataguard_stats
WHERE name IN ('transport lag', 'apply lag');
-- Pass: both values trending toward 0
-- Check 3: MRP in real-time mode (STANDBY)
SELECT process, status FROM v$managed_standby
WHERE process = 'MRP0';
-- Pass: WAIT_FOR_LOG (not WAIT_FOR_GAP)
-- Check 4: Protection level matches mode (PRIMARY)
SELECT protection_mode, protection_level FROM v$database;
-- Pass: both values are identical
-- Check 5: No archive gap (PRIMARY)
SELECT thread#, low_sequence#, high_sequence# FROM v$archive_gap;
-- Pass: no rows returned
-- Check 6: DGMGRL shows SUCCESS
-- dgmgrl /
-- SHOW CONFIGURATION;
-- Pass: Configuration Status = SUCCESS
10. Prevent MRP from Stopping Again
Configuration Hardening to Keep MRP Running
- Standby Redo Logs created: same size as online redo logs, groups + 1 per thread
- Data Guard Broker enabled with
dg_broker_start = TRUE on standby spfile
- Broker database state confirmed as
APPLY-ON in DGMGRL
- MRP startup trigger created (only if not using broker auto-restart)
- Protection mode set to Maximum Availability, not Maximum Protection, unless mandated
- Monitoring alert for zero MRP0 in v$managed_standby with 5-minute response SLA
- Apply lag alert at 30-second threshold, not 5 minutes
- DGMGRL
VALIDATE DATABASE scheduled weekly
-- MRP process health check (run on STANDBY)
SELECT CASE
WHEN COUNT(*) = 0
THEN 'CRITICAL: MRP not running, Data Guard apply stopped'
ELSE 'OK: MRP is running'
END AS mrp_health_status,
SYSDATE AS checked_at
FROM v$managed_standby
WHERE process = 'MRP0'
AND status NOT IN ('IDLE');
-- Apply lag alert
SELECT name,
value,
CASE
WHEN TO_DSINTERVAL(value) > INTERVAL '0 00:00:30' HOUR TO SECOND
THEN 'ALERT: ' || name || ' exceeds threshold'
ELSE 'OK'
END AS status
FROM v$dataguard_stats
WHERE name IN ('transport lag', 'apply lag')
ORDER BY name;
FAQ: MRP Process Not Running in Oracle Data Guard
Why is the MRP process not running in Oracle Data Guard?
The MRP process stops in Oracle Data Guard for five main reasons: manual cancellation during maintenance where the DBA forgets to restart it, missing Standby Redo Logs that prevent real-time apply from initialising, Data Guard Broker setting apply state to OFF, a protection mode conflict causing Oracle to abort MRP, or no auto-start configuration after a standby instance crash. Each cause requires a different fix command.
How do I restart the MRP process not running in Data Guard 19c?
Run on the standby: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION; If using Broker run in DGMGRL: EDIT DATABASE standby_db SET STATE='APPLY-ON'; Verify with: SELECT process, status FROM v$managed_standby WHERE process='MRP0'; Status should show WAIT_FOR_LOG or APPLYING.
What is the difference between MRP not running and ORA-16766?
They describe the same condition. ORA-16766 is the Oracle error code that appears in DGMGRL SHOW CONFIGURATION when MRP is stopped. MRP not running is the technical description of the underlying cause. Both mean the Managed Recovery Process has stopped and redo is no longer being applied to the standby datafiles.
Why does MRP stop immediately after I restart it?
Recurring MRP stop after restart has three causes. First, Data Guard Broker has apply set to OFF and overrides your manual SQL restart within minutes, so fix through DGMGRL instead. Second, missing Standby Redo Logs cause MRP to fail on start immediately, so create SRLs first. Third, Maximum Protection mode with an unstable standby network connection causes repeated MRP abort, so lower to Maximum Availability until the network is stable.
How do I prevent MRP from stopping after a standby reboot?
Create an AFTER STARTUP database trigger on the standby that checks the database role and runs ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT. Alternatively, enable Data Guard Broker with dg_broker_start=TRUE in the standby spfile and confirm the broker state is APPLY-ON, which makes broker auto-restart MRP after any restart automatically.
Dealing with a different Data Guard issue?
Drop a comment with your exact symptom and I will help you diagnose the root cause.
Comment DG
About the Author
Chetan Yadav
Chetan Yadav is a Senior Oracle, PostgreSQL, MySQL, and Cloud DBA, Oracle ACE Apprentice, with 15+ years of hands-on experience managing mission-critical production databases across finance, healthcare, and e-commerce sectors.
Chetan has resolved MRP stopped and ORA-16766 incidents across multiple production Oracle 19c environments, including RAC primaries with Active Data Guard standbys on dedicated WAN links. His writing focuses on exact diagnosis and exact fix commands backed by production experience.
This blog covers real-world DBA problems, cloud architecture, and practical learning, not theoretical documentation or vendor marketing.
No comments:
Post a Comment