Friday 30 August 2013

It's all a bit NATS...

Having incorporated NAV and COM frequencies in the iNavCalc results generated for a specified route, the issue of stale data needs to be considered. As such, I've been exploring ways of keeping the data up-to-date in the most efficient manner.

I've focused initially on the UK (where I fly). Encouraged by the fact that NATS (the company contracted to provide ATC services & associated products to the UK CAA) provide PDF files with up-to-date VFR frequencies for UK licensed airfields, I assumed they could make the underlying source data available in machine-readable form (database extracts via Excel or text comma-separated files, or better still, via on demand web--service calls). I asked them and they responded (quote):

"That's not a service NATS can offer"

Undeterred by this inital reaction, I made contact with the relevant management at NATS. Basically, that message was reiterated. Moreover, it seems to be the case that NATS *can't* (readily) offer that service, rather than simply *won't* offer such a service, even for a fee. I then enquired if this stance is perhaps because I was a "small player offering a free service via my apps etc", to which they responded "no, even the large commercial data-aggregators do not get such machine-readable data feeds from NATS. They simply download the PDF files and transcribe the values from those reports".

I find this astonishing. One has to assume that the data exists electronically in a database *somewhere* within NATS. Otherwise, how do they generate their PDF files ? Surely not by hand, manually transcribing from source records ?

NATS were courteous in their response, but the fact remains that it would appear to be the case that the UK frequency data cannot be made available in machine-readable form. That means that any individual (like me) or any organisation (like the major commercial data aggregators, or anyone in between) has to "page-scrape" the PDF files in order to retrieve the information. Apart from being inefficient, this approach is error-prone, as with any situation where humans are transcribing data which they read from one source document, and write (type) to another. Note: I use the term "page scrape" with reference to the (clunky) practice of writing a software routine which parses files intended as human-readable (eg., HTML, PDF etc). This is one crude workaround to the problem of not having the data in a fundamentally machine-readable form. However, I consider software "page scrapers" essentially equivalent to human "page scapers": both are inefficient and error prone. Software "page scrapers" are also particularly sensitive to changes in the underlying page structure.

Technically, it would be straightforward to implement a database extraction or, better still, a web-service to efficiently distribute the up-to-date data, machine-to-machine, minimising the possibility of human-error. I even tried to appeal to NATS on the basis of flight safety: just like MET and NOTAMS are considered vital to safe flight-planning, so is having the correct frequencies, it could be reasonably argued. Global MET and NOTAM data is made available, for free, in machine-readable form (iNavCalc and other such nav/planning apps make use of these electronic feeds, free of charge), so why not the frequency data ?

In fact, it used to be. See the following link for a description of the now-defunct DAFIF. Unfortunately, the service had to be withdrawn because, as I understand it, some contributing countries took issue (from a copyright perspective) with how the DAFIF data was being used.

...but it still leaves the question: for those countries not sensitive to copyright concerns (and I believe the UK is in that category), and who already make the data available for free to end-users via downloadable PDF files, why not make the underlying data available electronically ? Note: the same goes for *all* data available in the AIP, not just frequencies. If it is available in principle via PDF download, then why not in machine-readable (rather than just human-readable) form ?

In the meantime, and since I wanted iNavCalc to have the latest corrected data at least for the UK, I've gone ahead and "page scraped" the most recent VFR COM frequencies PDF files. So, next time you use iNavCalc for a UK route, you should see the most up-to-date COM frequencies reported (note: NAV frequencies still to be refreshed)...with the usual caveat that iNavCalc is unofficial and you must refer to original source data for navigation etc.

For all other global regions, if you notice an error in the iNavCalc outputs, please report via this form, and I'll undertake to make the corrections manually. Better still, if you somehow know of a reliable machine-readable source for your geographical region of interest, do please let me know, and I'll do what I can to engineer the appropriate automated interface to that source.

Note: NATS do offer a free email subscription service (to which I subscribe) whereby they provide monthly updates of all upcoming changes to the UK AIP. However, as with the PDF files, the information is written into the body of the emails in human-readable form, not machine-readable form. So although these emails make the transcription of updates quicker (building upon an existing current archive, say), they are subject to the same underlying issues of inefficiency and being prone to human error.

Wednesday 28 August 2013

Video Diary: Bulldog Mini-Tour of Scottish Highlands and Islands

In preparation for participating in the upcoming RAF Leuchars Airshow, we wanted to position our two ex-RAF Bulldog aircraft at Prestwick (EGPK). This should allows us an easy transit to Leuchars (EGQL) on 6 September 2013 (weather-permitting!).

One aircraft, xx546 (aka G-WINI) is based at Peterborough-Connington (EGSF). The other (mine) xx667 (aka G-BZFN) is based at Ronaldsway, Isle of Man (EGNS). So, the first port-of-call was Blackpool (EGNH), an airfield convenient to both of us. We met up there last Saturday morning (24 August 2013). Here's a picture of the two Bulldogs parked at Blackpool.

The next leg took us to Prestwick (EGPK). Here's the coastal route (in GoogleMaps, courtesy of iNavCalc RouteView) we followed on Saturday afternoon and here's a picture of the two aircraft parked-up at EGPK.

With a day of margin (Sunday 25 August), and excellent weather, we decided to embark on a mini tour of the Scottish Highlands and Islands. Here's the route we selected (in GoogleMaps, courtesy of iNavCalc RouteView), based on the clear skies in the west (by contrast, low cloud in the east, best avoided) and here are a few video clips taken on-board xx667 with a #GoPro Hero2 cockpit-mounted camera (click here for simple instructions on exporting video clips from GoPro to YouTube, hassle-free).

Formation-flying videos from cockpit-mounted GoPro

Stream takeoff (5 seconds apart) from Prestwick (EGPK)

Joining-up in formation after taking off from Prestwick (EGPK)

In formation over Mull Of Kintyre (and ex-RAF station "Macrihanish" EGEC -- now Campbeltown Airport)

In formation over Sound of Jura

In formation over the island of Scarba

In formation over the Firth Of Lorn

Breaking to land at Oban airport (EGEO)

Landing on runway 01 at Oban airport (EGEO)

...and here's a picture of the aircraft parked at Oban.

The second leg took us over Loch Awe Loch Fyne, Loch Lomond (including the summit of Ben Lomond), Helensburgh, Greenock, Rothesay, West Kilbride, Irvine, and back in to Prestwick. Unfortunately my GoPro WiFi pac had run out of charge by this time, so no videos possible :(...

Next stop, Leuchars :)

Saturday 17 August 2013

GoPro to YouTube

Want to upload your GoPro videos to YouTube ? Here's a simple step-by-step "how to" guide which works reliably with minimum hassle. Note: this guide is intended only for quickly getting your GoPro videos on to YouTube without any fancy editing (e.g., video effects, adding separate soundtracks etc). For such "fancy" editing etc, you will need to do more than presented here. 

Kit list

  • GoPro Hero2 (should also work with Hero3, but I haven't tried)
  • Windows 8 laptop (should also work with Windows 7 etc but I haven't tried)
  • Latest version of GoPro CineForm Pro software installed on Windows 8 laptop
  • Latest version of Microsoft Windows Movie Maker installed on Windows 8 laptop

Steps to follow

  1. Connect GoPro camera directly to laptop via USB, or remove the SD card from the camera and insert into an SD cardreader on laptop
  2. Use CineForm Pro to "Import & Convert" videos (or partial clips extracted from videos) from camera source video files. The built-in Users' Guide gives step-by-step instructions. This is a simple procedure, especially after doing it a couple of times.
  3. Use Windows Movie Maker to load each clip from result of previous step, and export ("Save Movie") via the built-in "YouTube" format option. Simple procedure.
  4. Upload each clip from result of previous step to YouTube.

There are of course many other combinations of platforms, tools and procedures to achieve the same result, but in my opinion the above approach is the best and simplest, based on my own recent experiments with various approaches. So far, I've never had an upload rejected by YouTube when I've followed these steps. By contrast, previous attempts via different paths also worked but often led to (frustrating) rejections from YouTube.

Hope you might find this useful.

Sunday 4 August 2013

FlyLogical goes cloud mad, embraces Total Internet

The thousands of iNavCalc users will know that FlyLogical makes extensive use of internet-based resources when compiling PLOGs, MET reports, NOTAMs, etc. Specifically, the following core internet-based services are employed:

  • Email
  • Web
  • GoogleMaps
  • MET via web-service
  • NOTAMs via web-service
  • Twitter

All of the above services rely wholly on the internet, and to a significant extent, on "the cloud", but until recently, the actual FlyLogical servers pulling everything together were physical machines operated by FlyLogical from a datacentre ("computer room") quite literally "down the road".

As of midnight last Saturday (27 July 2013), all that has changed: the entire FlyLogical server infrastructure was moved to "the cloud". This means that FlyLogical has no ownership, low-level control, or even knowledge of the location of the actual machines which now host the apps. Instead, FlyLogical rents virtual server "space", on-demand, from a "cloud provider".

The reason for moving to the cloud was primarily to improve resilience. The cloud provider can ensure that the servers  are "up" with significantly greater reliability than the previous approach. For example, during the Leuchars Airshow last year, it just so happened that the FlyLogical physical servers went offline due to a lightning strike on the building housing the datacentre. Such rare events can -- and did-- happen. This was inconvenient for users, not least for me, since I was relying on iNavCalc for planning my trip back from Leuchars!

Apart from resilience, the cloud also offers improved performance in terms of screen-response (on the web browser for the web-apps) and email turnaround time (for PLOGS etc generated via the iNavCalc mobile-, email-, and web apps).

I'm hoping that nobody even noticed the switchover last weekend when the FlyLogical apps were "down" for a mere few seconds during the transition.

With this migration of the entire computing infrastructure to the cloud, I've embraced (what I like to think of as) the Total Internet.