I think the picture pretty much describes everything you need to know. Basically, you freeze some plain black coffee in an ice cube tray. Pour some milk into a cup and pop in a couple of frozen coffee cubes. As the cubes melt, the coffee will float on top so every time you take a sip you'll be guaranteed some coffee flavour!
I used chocolate milk for mine but you can use skim, almond, whole, chocolate, 1%, 2%, or soy. Whatever floats your boat.