FWIW here is the method I use and have been doing so for over 20 years. I have a half pencil, a pencil planed into a semi-circle for it's whole length cutting it right in half so that a pointed end protrudes out dead flat. The pencil is now a straight edge with a point. I install the nut and then mark a line with the pencil by sliding the pencil across the frets as it marks the nut face.
So in theory the very bottom of the graphite pencil line is the same height of the top of the frets they were marked from. In practice this is not the case but it's close enough to rough out the slot depths very quickly. After that I use the method Frank Ford outlined and just do it by eye. I like to taper the heights from the Low-E to the High-E such that there is clearance at the low end and on down to the treble strings which just graze the first fret when holding the string down. I do this because it seems to me that a lot of players like to pound on an open E chord for example and it gives a bit more room. Plus back buzz can be a problem on a less than ideal fretboard.
I've tried using feeler gauges (Yes I am a Yankee and have always used a 'U' but maybe that's because I'm a NEw England Yankee close on up there to Canada
) but for some reason I lack the ability to use them. Or at least I just don't trust myself measuring with them as they seem to me to be too sensitive and reveal different results every time.