Well, time for full disclosure. I will confess my biggest mistake yet in order to (hopefully) save someone else the problem in the future.
I pulled my engine today and pulled the crank out. When I got the crank out, I noticed that the rear main bearing was showing some wear also. In addition, the rear main seal was looking pretty bad.....as in bad enough you would think it had 5000 hours on it, not 5. As I was sitting there looking at them, my brother-in-law (a tractor mechanic) pointed at the rear main bearing and said, "Look at that, the bearing is made wrong. It has the oil hole in the wrong location and there is no place for the oil to get to the crank."
Just then, my world started spinning and I started to get a little light-headed. I thought back to several topics I read on this very website regarding installing main bearings. Of course, I did not find this forum and read these topics until months AFTER I had rebuilt my motor. What were these postings and topics about you ask???
Well, they had to do with installing the rear main bearing and reversing the top and bottom bearing.
I slowly walked over to the rear main cap and looked inside. Sure enough, that one also had the "oil hole in the wrong location." I had made a bonehead mistake when I put the crank in and reversed the top and bottom bearings.

Now it made sense why my #4 rod bearing was ate up and the others were not that bad. It also made sense why my oil pressure always seemed high (around 60lbs). Oil was stopped right behind the oil pressure gage outlet and not getting to #3 main bearing and #4 rod bearing. I had just assumed it was because it was a fresh rebuild. My hard starting when warm could even be related to this. If enough heat was building up back there, I would assume the bearings and crank were expanding and making a pretty tight fit....causing the engine to be harder to turn over when warm. Heck, it may even be the reason it is hard to shift into 2nd gear. (Ok, probably not but I guy can dream.)
So there you go. I'm a big dumb animal. When I built the motor in March of 2008, I was going through a rough patch in my life but it is still an important detail I should have paid more attention to.
As in everything in life, there is a reason for things happening when and were they do. I am thankful the Lord decided to point out my mistake to me while the jeep was sitting in my garage and not in the middle of a parade or trail ride or car show. I'm pretty sure if that happened I would not be getting out with just turning the crank and replacing some bearings.

I'm also thankful to everyone on here who convinced me to drop the pan and pull a rod cap to check things out. While the fuel pump may not have been the root cause, it definitely started me down the path that led me here.
My opinion of Federal Mogul bearings has been restored also. Obviously, they are tough enough to run for a pretty good while.....even when they are not getting any lubrication.

Lets see where the next road takes me!