Hello there,
I remember in older versions of OTRS, Escalation notification recipients were governed by any of the following two text-based configurations:
# notify all agents who selected the queue in "my queues/custom queues"
Module => 'Kernel::System::GenericAgent::NotifyAgentGroupOfCustomQueue',
# notify all agents who can access the ticket with rw permissions
Module => 'Kernel::System::GenericAgent::NotifyAgentGroupWithWritePermission',
Now it seems like the GenericAgent has nothing to do with this and instead there is the following path:
Kernel::System::Console::Command::Maint::Ticket::EscalationCheck