So: in mid-January I posted the following challenge to some technically capable friends on Facebook:

---

I’m looking for a small set of geeks who can help me with a DNS metrics experiment. Must haves:

• understanding of shell scripts & DNS
• availability of “dig” tool
• fast Linux or MacOS host
• solid local network without significant droppage
• solid Internet link of known and reliable bandwidth
• known and reliable upstream DNS resolver which won’t fall over and is unlikely to rate-limit you if you hammer it a bit

I’m developing a test script, not yet finished, but I will need beta-testers and full-testers. Summary: script performs 5 batches of 100x reverse-lookups on random, possibly invalid, IPv4 under a stopwatch. Some might fret about that.

---

The reason for this bizarre request was – under pressure of time – to attempt to create a rough but globally neutral experiment that would test the latency of DoHoT against that of “normal, domestic” DNS installations. My goal was explicitly to avoid the benefits of cached responses and to force the exercise of complete end-to-end query resolution, and to deliver this test in a rush because of the upcoming NDSS conference.

So I wrote a script which sought some basic information from the user, and then generated five separate, distinct sub-shell-scripts, each of which made separate invocations of dig to reverse-lookup 100 randomly-generated IPv4 addresses – one invocation per address – measuring the elapsed time for each sub-script’s execution. No attempt was/is made to assure the validity of the IP addresses, other than to constrain each octet to a random number in the range 1..254 inclusive.

I then shared the script with volunteers who ran it, and some ran it in several environments; each volunteer is coded (e.g. “am1” is myself, “am2” is another person with the same initials) and each run by a volunteer is given a sequence letter (“a” = 1, “b” = 2, …)

The value of this experiment is limited but far from worthless; most discussions of DNS performance pivot upon metrics like “request latency in milliseconds” or “page load latency“, and/or “resolve the Alexa top 1000 domains“, but here we are attempting to build a picture of neutral outright performance – “here are 500 random IP addresses, they may not even be legal, go look them up backwards, one at a time, one process at a time, on whatever hardware and platform you have available“.

Doubtless there are flaws but with this metric DoHoT is not too bad; clearly it’s at the slower end of DNS service provision but it’s within the top 70-percentile of the submitted results, and with far greater privacy guarantees than any other available DNS offering.

It will be necessary to take many more measurements to extract meaningful rankings, and of course there are many other possibilities like cache-warming and cache-tuning which could improve the DoHoT user experience.

The bottom line, both of this and of my upcoming paper, is this: if you are running DNS over HTTPS over Tor (DoHoT) then you are not actually having a dreadful experience; and maybe you will appreciate the additional privacy benefits more, on that basis.

footnote: volunteer “dr1a” apparently forwards their queries from their stub, direct to one of the DNS root servers; this is what they mean by “myself”.