It would be good if you got to see him more also, because right now you think he is 'perfect' (for you anyway right?), but before crossing over the line of friendship (since he's been a family friend for so long) maybe you ought to think about this a little bit more heavily.
My usual advice is to go for it, because living with "what if"s is hell! I'm not telling you to wait forever or anything like that, just maybe hang out with him a lot more see if you think it would work out. Also can you face the consequences if it doesn't? Say that you do go out with him, and things do not work out... are you willing to give up this great friendship if you have a bad break up? Once the line is crossed, it can be hard to bring things back to a great friendship level.
But, the main thing is, after weighing consequences, do what you feel is right. The only way to know how he feels is to ask him! You seem to be getting pretty good vibes that these feelings are mutual, so if you feel it's right, 'fess up that you like him, and ask if he feels the same.
About the 3rd musketeer that also likes you... Well, life is full of hard breaks, and it's awful to break someone's heart or to get your heart broken. Keep his feelings in mind, but you can't let him stop you from being with who you truly love. He may just have an innocent crush on you and be happy that things worked out between you two.
But you'll also have to be prepared for the other side of that coin. Maybe the friendship with him will be hurt by it. It happens. Keep that in mind when weighing out the consequences, and act according to your priorities. As the saying goes: You can't please everyone, sometimes you've just got to please yourself.
Best wishes to you in how this all turns out,