I can't figure out who James looks like, either, and it's really starting to bug me.
Until recently, I thought he really resembled the Rhys-Jones side of the family. But his face seems to have changed quite a bit lately (I almost didn't recognize him in some of the pictures from today), and I swear that pretty much the only family resemblance I can see is that James' ears look like Edward's, which I've always thought was the case. Genetics are so funny. Louise has always looked SO MUCH like her dad, and as she's gotten older I can occasionally see a slight resemblance to Sophie, but I can't think of anyone on either side of the family who James looks like at the moment. (Of course, I don't know what Sophie's parents looked like when they were young or anything like that.)