So I spent some time this weekend to try and show where 5 12 hour days went in analyzing data. This first graph shows the number of successful mirror requests and breaks down the 503 error's per server.
 |
| All mirror requests since 2018-01 |
The two drops are where logs failed to be copied to the central system versus a problem with the mirrors. You can see that Monday through Friday, Fedora sees a lot of requests and then Saturday and Sunday we see a dip. You can also see that we have gotten an increase in usage since November. The tiny area at the bottom is the number of 503's.. which kind of makes it look unimportant. [Unless you are doing a lot of builds and keep running into it.]
 |
| Just the 503's sir. |
The above is just a graph of the 503's broken down by each server which sees them. We see that in January, we see a large increase of 503's on 2 servers. The first one is proxy11 which is in Europe and the server may be underpowered for what we are needing it to do. The second was on proxy01 which a lot of sites have hard-coded. If the above graph was done by the minute, you would see it as many many tiny spikes at :02 -> :10 minutes after the hour and most of the day empty.
The graphs go to 2019-02-15 and the last 4 days have shown a decrease in 503's but proxy01 and proxy11 are still having several thousand a day. I am still looking at other fixes we can do to make this a less painful experience for people when the top of the hour occurs.
No comments:
Post a Comment