2024-06-28

What happens to EPEL-7 when EL-7 goes EOL.

What happens to EPEL-7 when EL-7 goes EOL

What is happening to RHEL-7

The Red Hat Enterprise Linux release cycle is fairly straight forward with releases lasting about 10 years after their initial release.

RHEL ver Release Date EoMaint EoExtended
5 2007-03-15 2017-03-31 2020-11-30
6 2010-11-10 2020-11-30 2024-06-30
7 2014-06-10 2024-06-30 2028-06-30
8 2019-05-07 2029-05-?? 2031-05-??
9 2022-11-08 2032-05-?? 2034-05-??
10 2025-??-?? 2035-??-?? 2037-??-??

On June 30, 2024, Red Hat will stop doing general maintenance support of RHEL-7 and no more updates to that operating system will be available without purchasing ‘Extended Life-cycle Support’ contracts from Red Hat or similar contracts from SuSE’s Liberty Linux, Perforce’s OpenLogic, or various other consultants and companies offering various services.

What is happening to EPEL-7

The Fedora Extra Packages for Enterprise Linux (EPEL) is a repository of additional packages built against the Red Hat Enterprise Linux (RHEL) releases. When a RHEL release leaves maintenance mode and moves into ‘Extended Life-cycle Support’, the EPEL package set corresponding to that is closed and archived. No new builds or updates are done against that particular release, all bugs related to that release are closed as CANTFIX, and most mirrors will remove the content after the master mirror moves items to archives.

The usual process for closing out an EPEL release depends on how much work the central Fedora release engineering team have to deal with at the time of the RHEL EOL. In cases where a Fedora release is needing a high amount of focus, the only actions may be that new builds will be turned off on the day of the EOL, and then later other steps will be worked out. In other cases, all the actions occur fairly close together.

  1. A reminder email that the EOL is going to occur is sent to various Fedora mailing lists like: devel-announce@lists.fedoraproject.org and epel-announce@lists.fedoraproject.org. System administrators using EPEL should be following at least the second email address to see what issues may be occurring with the repository.
  2. The Fedora Build system is a complicated tool-set and so Release Engineering have to disable the building and composing of that particular EPEL release in multiple places. Because the release cycle is so long for EPEL compared to Fedora releases, there may be inconsistencies in how things were named or done. Basically it is a slower process than with Fedora releases to make sure that the tools continue to work after an EOL.
  3. All bugzilla.redhat.com bugs related to that release will be closed with a short stock answer tailored for that release as ‘CANTFIX’.
  4. The release is now copied into the Fedora down-loaders /pub/archive/epel tree with an appropriate name like /pub/archive/epel/7.9.2024-07-01 and then symlinks are made. A week is given for the archive mirrors to catch up with this new directory tree.
  5. The Fedora mirrormanager will be changed to point requests for ‘repo=epel-7’ to the mirrors which have /pub/archive.
  6. The content then will be removed from the main mirror locations of /pub/epel/ and mirrors who rsync will catch up to that in a day or so.

Usually about a week to a month after this happens, either the mailing lists or IRC will get people wondering where the content went. However in the current age of Continuous Integration and Container builds, changes in the EPEL-7 directory structure seem to cause reactions hours after they occur.

How to deal with this EOL.

The main advice I have for system administrators dealing with an End of Life operating system is the following:

  1. Mirror the content locally from whatever mirrors you can find. This lowers network bandwidth usage, and deals with the vagaries of archive sites removing content because they no longer can handle the load from large numbers of unexpected users.
  2. Work with your management on how to deal with the technical load that these systems are going to cause.
    1. Budget out what extended maintenance costs for critical security fixes to services like web-servers, databases, remote logins, etc are going to take.
    2. Budget out what mitigations are going to be required to keep these systems from causing longer term headaches. This may be extra firewalls or additional staff needed to take over software maintenance.
    3. Budget out what replacements of the hardware and software to move to an operating system which has long term maintenance support.
    4. Actually this is a very long list and probably should be its own blog post.

End Thoughts

Dealing with End of Life software is rarely an easy task. Just realize that you aren’t alone and there are tens of millions of systems which are going to be ‘unsupported’ on July 1st. Be patient with yourself and your infrastructure as you deal with this. Good luck.

2024-06-20

Updates on the CentOS 7 EOL: 2024-06-30

I am writing this on 2024-06-20 which means according to the IRC centbot:

  centbot: CentOS 7 will go EOL on 30 June, 2024 -- in 1 week,
 10 hours, 22 minutes, and 41 seconds.
  

The CentOS Systems Administrator, Fabian Arrotin, has posted several updates about what to expect starting Monday Jul 1st, 2024. The first email covers what services will stop after the 'End of Life' occurs:

  1. mirrorlist.centos.org will be decommissioned. This will affect any system trying to do a `yum update` or similar command after that date.
  2. forums.centos.org
  3. will be turned off and decommissioned. This was a volunteer effort by various system administrators who are ending their work with CentOS at the end of 7.
  4. torrent.centos.org
  5. will be turned off and decommissioned. This was useful for putting out new isos, but was no longer done for the CentOS Stream project.

Removal of mirrorlist will probaby cause various cron jobs and similar tools to stop working. Manual work will be needed to either turn off the CentOS repos affected in /etc/yum.repos.d/ files or something similar. The loss of forums will mainly affect people looking for answers for problems with EL7 and before systems. Similar answers may be available at stackoverflow or similar sites.

Fabian's second email covers what systems mirroring CentOS Linux 7 can expect and when.

  1. First it reiterates that all EL7 content from mirror.centos.org will be removed. Also that the mirrorlists will be turned off and no longer answering.
  2. Second crawlers will be turned off to see what mirrors are 'up-to-date' with EL7 content as the release is over.
  3. Third, the head rsync mirrors will continue to exist, but will only have an 'empty' filelist on it. This will cause all mirrors which do an rsync with --delete flags to remove all EL7 content from their mirror. This is normal for end of life releases as disk space needs to be used for other content.
The biggest issues that system administrators with EL7 systems need to be ready to do are the following:
  1. Be prepared that existing scripts and tools pulling in EL7 content will break without work.
  2. If you are mirroring EL7 content, and need to keep it, you need to check that the rsync scripts do not delete content which is no longer on the mirror. If you are using reposync or similar commands you will need to look for similar options.
  3. You should look at mirroring the content local to your site until your organization has a transition plan to another operating system.
  4. You should work with your management and financial resources on a transition plan if you haven't already.