Recent EPEL dnf problems with some EL8 systems

Last week there were several reports about various systems having problems with EPEL-8 repositories. The problems started shortly after various systems in Fedora had been updated from Fedora Linux 38 to Fedora 40, but the problems were not happening to all EL-8 systems. The problems were root-caused to the following:

  1. Fedora 40 had createrepo-1.0 installed which defaults to using zstd compression for various repositories.
  2. The EL8 systems which were working had some version of libsolv-0.7.20 installed which links against libzstd and so works.
  3. The EL8 systems which did not work either had versions of libsolv before 0.7.20 (and were any EL before 8.4) OR they had versions of libsolv-0.7.22

The newer version of libsolv was traced down to coming from repositories of either Red Hat Satellite or a related tool pulp, and was a rebuild of the EL9 package. However it wasn't a complete rebuild as the EL9 version has libzstd support, but the EL8 rebuild did not. 

A band-aid fix was made by Fedora Release Engineering to have EPEL repositories use the xz compression method for various files in /repodata/ versus libzstd. This is a slower compression method, but is possible to be used with all of the versions of libsolv reported in the various bugs and issue trackers.

This is a band-aid fix because users will run into this with any other repositories using the newer createrepo. A fuller fix will require affected systems to do one of the following:

  1. If the system is either Red Hat Enterprise Linux, Rocky Linux, Alma Linux, or Oracle Enterprise Linux and not running EL-8.9 or later, they need to upgrade to that OR point to an older version of a repository in https://dl.fedoraproject.org/pub/archive/epel/
  2. If the system has versions of libsolv-0.7.22 without libzstd support, they should contact the repository to see if a rebuild with libzstd support can be made. 
  3. If the system is CentOS Stream 8, then you should be making plans to upgrade to a different operating system as that version will be EOL on May 31st 2024.

External References


No comments: