I suspect that you didn't notice that there are two sets of loading data for the .308 Win; one for the standard bolt rifles and one for the service rifles. You obviously were incorrectly looking at the bolt rifle data while I was looking at the service rifle data since that's what the OP asked about. Now that I've figured out that you are looking at the wrong data I see the IMR 3031 load information.
The first problem is that you are using the Hodgdon data for an M1A service rifle. Hodgdon isn't all that reliable for most of their load data in my opinion but their .308 Service Rifle data is absolutely wrong. They show that their service rifle data was developed using a 24 inch barrel, I don't know what 7.62x51mm or .308 Winchester service rifle they used but it could not have been an M1A since they've never made an M1A with a 24 inch barrel. Hornady, on the other hand, developed their load data using an M1A rifle. Comparing the data from each will prove that Hodgdon is completely unreliable for M1A data.
The bullets used in each manual are different so the data isn't a perfect comparison but it's pretty close if you compare a Hornady 168gr Amax to a Sierra 168gr HPBT. Using those bullets and IMR 4064 (neither Hodgdon nor Hornady show IMR 3031 with this weight bullet so I used IMR 4064 since both show load for it);
| Min Powder Charge | Maximum Powder Charge |
Hodgdon Manual | 41.5gr 2518 FPS 43,800 PSI | 43.9gr 2653 FPS 52,000 PSI |
Hornady Manual | 36.1gr 2100 FPS | 41.6gr 2500 FPS |
It's obvious that Hodgdon's data is very different from data that was developed by testing in a M1A rifle. What's more telling is that if you check out Sierra's data it's very close to Hornady's data (taking in to consideration that Sierra used a 24 inch barrel for their testing). Note that the max powder charge in Hornady's manual is almost the same as the min powder charge in Hodgdon's manual. It is common to find that the M1A rifles will produce a muzzle velocity of around 2550 FPS using a 168gr bullet when you use a powder that is well suited to the design of the rifle's action. So Hornady's 41.6gr load is very close to that traditional combination. And while you might want to claim that Hodgdon says that that is a good powder charge too but then says that you can go up from there, you have to remember that Hodgdon didn't use an M1A for testing and that over the years many M1A shooters have found that anything more than 2550 FPS with a 168gr or heavier bullet is not good for the rifle. Long range National Match shooters will push their rifles with hotter loads but they also replace barrels quite often and they actually do see the receivers crack from time to time.
The next problem is that the Hodgdon data used the Barnes TTSX and the Sierra HPBT bullets for their bolt gun data and just the Sierra HPBT bullet for their Service Rifle data, neither bullet is very similar to the Swift bullet. The Swift bullet is actually a 165gr bullet that is much shorter than the other bullets. That means that the pressures and velocities shown in the Hodgdon data isn't going to match up very well with the bullet that the OP wants to use.
While CFE223 does produce higher velocity at lower max pressure (under the TTSX bullet) it also produces higher port pressure for the M1A rifle which will batter the action and create excessive wear compared to IMR 3031. In fact, I would never recommend using 41.5gr of IMR 3031 under the TTSX bullet because the max pressure will actually be higher than Hodgdon claims and it will definitely create premature case failure and excessive wear on the rifle's action. I recommended a max powder charge of 41.5gr of IMR 3031 only because the OP is using the Swift bullet and its shorter length provides more usable volume in the case and the pressures will be lower.
And finally, I've tried using CFE223 in my M1A rifles and can honestly say that it didn't produce the kind of accuracy/precision that I get from IMR 3031. So between my personal experience and pressure data that I believe is accurate, CFE223 is NOT a good choice for an M1A rifle.