I got pretty deep into this. I had communications with Performancelinebearings.com (Paul is awesome). After sending some photos we decided everything looked right. Many people told me it was normal, no one knew anything else to tell me, and I finally gave up. But guess what?
It's FIXED! I KNEW it was too darn loud, there's just no way it should sound like that! I wanted to sell the bike...
I started having some shifting problems and I couldn't get it properly indexed, so I took it to my mechanic. He found a minor cable routing issue, but most importantly he found that the chain was too long. He took out some links, and in addition to fixing my indexing issues, the SES pulley quieted by at least 75%. Apparently there wasn't enough tension on the chain and for whatever reason that was causing the excessive noise.
I can still hear that pulley a little, but NOTHING like before. I also do notice it gets a bit louder after a couple hours of riding so regular cleaning and lubrication is recommended (and Paul said not to use dry lube on bikes with SES, so I switched to Finish Line Wet).
I can't tell you how happy I am to have this figured out, it was really driving me nuts.
Thanks for this info, I will check my chain length.