It's all a bit NATS...

Having incorporated NAV and COM frequencies in the iNavCalc results generated for a specified route, the issue of stale data needs to be considered. As such, I've been exploring ways of keeping the data up-to-date in the most efficient manner.

I've focused initially on the UK (where I fly). Encouraged by the fact that NATS (the company contracted to provide ATC services & associated products to the UK CAA) provide PDF files with up-to-date VFR frequencies for UK licensed airfields, I assumed they could make the underlying source data available in machine-readable form (database extracts via Excel or text comma-separated files, or better still, via on demand web--service calls). I asked them and they responded (quote):

"That's not a service NATS can offer"

Undeterred by this inital reaction, I made contact with the relevant management at NATS. Basically, that message was reiterated. Moreover, it seems to be the case that NATS *can't* (readily) offer that service, rather than simply *won't* offer such a service, even for a fee. I then enquired if this stance is perhaps because I was a "small player offering a free service via my apps etc", to which they responded "no, even the large commercial data-aggregators do not get such machine-readable data feeds from NATS. They simply download the PDF files and transcribe the values from those reports".

I find this astonishing. One has to assume that the data exists electronically in a database *somewhere* within NATS. Otherwise, how do they generate their PDF files ? Surely not by hand, manually transcribing from source records ?

NATS were courteous in their response, but the fact remains that it would appear to be the case that the UK frequency data cannot be made available in machine-readable form. That means that any individual (like me) or any organisation (like the major commercial data aggregators, or anyone in between) has to "page-scrape" the PDF files in order to retrieve the information. Apart from being inefficient, this approach is error-prone, as with any situation where humans are transcribing data which they read from one source document, and write (type) to another. Note: I use the term "page scrape" with reference to the (clunky) practice of writing a software routine which parses files intended as human-readable (eg., HTML, PDF etc). This is one crude workaround to the problem of not having the data in a fundamentally machine-readable form. However, I consider software "page scrapers" essentially equivalent to human "page scapers": both are inefficient and error prone. Software "page scrapers" are also particularly sensitive to changes in the underlying page structure.

Technically, it would be straightforward to implement a database extraction or, better still, a web-service to efficiently distribute the up-to-date data, machine-to-machine, minimising the possibility of human-error. I even tried to appeal to NATS on the basis of flight safety: just like MET and NOTAMS are considered vital to safe flight-planning, so is having the correct frequencies, it could be reasonably argued. Global MET and NOTAM data is made available, for free, in machine-readable form (iNavCalc and other such nav/planning apps make use of these electronic feeds, free of charge), so why not the frequency data ?

In fact, it used to be. See the following link for a description of the now-defunct DAFIF. Unfortunately, the service had to be withdrawn because, as I understand it, some contributing countries took issue (from a copyright perspective) with how the DAFIF data was being used.

...but it still leaves the question: for those countries not sensitive to copyright concerns (and I believe the UK is in that category), and who already make the data available for free to end-users via downloadable PDF files, why not make the underlying data available electronically ? Note: the same goes for *all* data available in the AIP, not just frequencies. If it is available in principle via PDF download, then why not in machine-readable (rather than just human-readable) form ?

In the meantime, and since I wanted iNavCalc to have the latest corrected data at least for the UK, I've gone ahead and "page scraped" the most recent VFR COM frequencies PDF files. So, next time you use iNavCalc for a UK route, you should see the most up-to-date COM frequencies reported (note: NAV frequencies still to be refreshed)...with the usual caveat that iNavCalc is unofficial and you must refer to original source data for navigation etc.

For all other global regions, if you notice an error in the iNavCalc outputs, please report via this form, and I'll undertake to make the corrections manually. Better still, if you somehow know of a reliable machine-readable source for your geographical region of interest, do please let me know, and I'll do what I can to engineer the appropriate automated interface to that source.

Note: NATS do offer a free email subscription service (to which I subscribe) whereby they provide monthly updates of all upcoming changes to the UK AIP. However, as with the PDF files, the information is written into the body of the emails in human-readable form, not machine-readable form. So although these emails make the transcription of updates quicker (building upon an existing current archive, say), they are subject to the same underlying issues of inefficiency and being prone to human error.