Monday, September 9, 2013

Summary, and technical discussion of 3D Earth Route Mapper

AUDIO NARRATION:



Hello all!  After a very long time in which my Around the World Trip blog lay dormant, it will be revived once more for this final summary post, and technical discussion about my favorite aspect of the entire trip.

If you are just seeing this chronology of my wonderful adventure for the first time, I welcome you, and thank you for visiting, and invite you to read from the earliest post, if you want to enjoy the journey with me virtually through my thoughts and images.  Boring for a lot of people who don't actually know or care about me, no doubt, but I did try to keep the commentary interesting, and discipline myself to document it fairly well, so hopefully maybe it could even be enjoyable for a complete stranger.

As a cautionary note, I would just warn any reader that I have always been a tad toward the... shall I say... excessively opinionated side.  Particularly as I mine my recollections, chaining them deeper and deeper back in time, I certainly observe that the downward trend in the degree to which I remain excessively opinionated, for which I have striven so long with significant success, nevertheless inevitably requires that my past self is always somewhat more opinionated and less refined than I would prefer.  Yet such are the benefits of continuing to discipline myself more and more to express myself properly and with sincere grace, that I must continue to try to improve, even though it will doom me to ever look back upon my past with some mild twinge of regret, knowing who I was, certainly compared to who I am now, and especially compared to who I someday hope to be as well!  So with that prologue, I now formerly disclaim, that any offensive remarks I may make toward anyone, are meant in sincere good will alone, and do not reflect the opinions of anyone other than my own self, or at least a version of myself several years ago displaced in time, which is actually always somewhat different than the present self, as I noted!  :)  Please therefore do forgive any ignorance you may detect in my verbiage or choices, attitudes, rants, raves, ramblings, or reconnaissance, for that matter...  ;)

For those of you primarily interested in the story of my journey, please do continue now, and divert your attention to the first post in the blog series, starting on April 26, 2011!

For those who already read my blog, or are interested in the technical discussion primarily, I now invite you to hear about something I really enjoyed, which was divorced from the actual content of the trip largely, but extremely instrumental, and inspirational, to help me create it.

I have always dreamed of circling the Earth, since I first learned it was a sphere.  When I was so fortunate to travel abroad my first time, I was deeply fascinated with the geometric aspect of the journey in particular, and the fact that our upright position was now many degrees rotated in both planes from our prior position.  (Ultimately that has led to many long hours of introspection as well about our true position over time, marveling at the fact that we are now roughly in the position relative to the galactic center that our progenitors were 250 million years ago, when Carlsbad Caverns was first being formed under a low inland sea, and further, that in fact, our galaxy has moved many tens or even hundreds of trillions of miles during that time as well, relative even to our nearest substantial neighbor in our local galactic group.)  So...  that seed of inspiration at a tender age was really the genesis of my long held dream of circling the entire globe.  That dream evolved later to prefer especially a Great Circle route, which actually approximates a full diameter of the sphere, as closely as possible.

So when I finally became able to afford such a trip, and found a suitable partner willing to undertake such an endeavor, with whom I could get along passably well enough to endure it, and fool-hearty enough to consent to attend with me, I began to plan!  I could never have afforded the business class tickets I required (never being the kind to enjoy camping whatsoever, and rather requiring very delicately controlled conditions for any sleep at all! hahaha), for such an extravagant journey.  So I was bound to use the cheaters way to do the trip, and use airline miles I was so fortunate to accumulate due to my current locale in Dallas, TX, primary hub of one of the largest airline alliances on the planet.  And that is what I did.

But...  upon finally accumulating nearly enough miles, when I actually began to try to start booking, I was deeply dismayed!  The friendly agent who greeted me on my initial naive phone call, was so kind, to spend an hour at least with me, discussing options, and alternate routes.  I quickly discovered it would be nearly impossible to achieve, particularly since the business class miles seats on coveted international routes are frequently sold out within minutes of becoming available, as much as a year in advance of their actual date of travel!

Further complicating my arduous task, was the absolutely imperative requirement that as many flights as possible be on what I consider "extreme" passenger aircraft.  Another aspect of my first flight overseas at a very impressionable age, which planted a seed that has grown into a monster of Jack and the Beanstalk proportions, was the utter shock and awe I felt upon seeing my first 747 parked at the gate, and walking up to it in the jet bridge.  I realized this was much larger than the largest house in which I had ever lived, and was going to lift gently and safely off the ground, transport us a fifth of the way around the surface of our (at the time I thought) massive planet, and touch down like a feather, allowing us to walk off in the same clothes in which we boarded, having completed a journey that in most prior days of human history, would have taken months, or years, and been very possibly the cause of one's demise!  This splendid fact gave me hope that anything at all must be possible, somehow...  And, my deep infatuation with those aircraft has never decreased since.  Of course, since that time in my childhood, now the A380 is happily added to the list of aircraft meeting the requirements I dictated for the monikers "extreme" and "commercial" to apply, though to my deepest sorrow, neither the Concorde, nor any other supersonic passenger aircraft, are any longer operational, and have therefore sadly fallen off that list.  So my list of favored aircraft for the journey was limited in fact, to two primary vehicles (or lines of them, since in fact there are several models of each), the 747, and the A380.  One small additional constraint, was that I must ride in the coveted bubble in the 747s.  In fact I was fortunate that all of the airlines on which I flew place Business Class seats in that hallowed space, but, when I was a child and first flew Al Italia, in business class, since it was just me and my brother and my Mom, I do remember we were up front.  Actually this is now considered the superior location by many long haul connoisseurs, and has been pretty much universally made the first class cabin on 747 flights to my knowledge (at least in the OneWorld network).  But I remember after we landed in Milan for a stopover before going on to Rome, when the other passengers exited, the nice flight attendant let me go up the stairs to see the posh exclusive space!  It was like being shown the riches of Ali Baba's cave, to my mind, and so, I developed a perhaps unhealthy, but absolutely rock solid obsession with riding in that space, for the remainder of my youth!  And so...  that additional constraint was obviously not negotiable.  I was riding in that bubble!!

All these constraints obviously could be very difficult to achieve.  Even the primary initial constraint, just getting flights to even meet the requirements to achieve the OneWorld Explorer award at all, was going to be a massive undertaking.  Adding in my other constraints, it seemed truly impossible, at first!  But, due to my excessive focus and drive at times, and the degree of my possibly unhealthy obsession, I could not consider that as a cause to stop my Don Juan style pursuit of this windmill that was my dream journey...  I proceeded to dream despite the obvious obstacles that were not seemingly passable.

And it was at that point, frustratedly gazing into this beautiful, vast lattice of knowledge and data, a crystal ball of sorts, to which we currently refer as the "Internet", that I happened one day across a unique tool, useful to aid me in discovering those flights matching my very unique and specific criteria.  Some kind gentleman, known to me only as "cockpitvisit" from the Flyer Talk forums, has published his work freely!  His tool is the Airline Route Mapper, and its author is the true prior art from which my utility is entirely derivative, so to him I am eternally grateful.  Should my endeavor actually ever produce any revenue (LOL) I would surely be indebted to him for a royalty, since my subsequent art includes his generously provided data cache, and a notable design feature.  However, those legal complexities shall just as surely never arise in this case, since this particular tool is of such limited scope and commercial value that it may in fact apply to exactly only one consumer, namely, myself!

Yet it does remain something of which I am exceedingly proud, and was also very beneficial and useful, outside of my own inane pursuit of a childhood fantasy/obsession!

The primary benefits of the prior art were that it contained a fairly accurate, updated, and comprehensive database of the entire flight schedules of the global airline industry, and implemented as well a fantastic search feature, with filters for aircraft type code, as well as airline, or even alliance.  These provided nearly exactly the functionality I required for my own planning.  And so I proceeded, thrilled that I had discovered an answer to my difficult dilemma!

This initial searching with cockpitvisit's tool did begin to yield some viable possible routes, on the OneWorld network, primarily on my desired aircraft types, which would bring me round the globe.  I began to get excited!  However, I quickly realized that if I really wanted to make my dream come true, as completely as possible, I would require the route to be roughly spherical.  As you will recall from any time you have seen some map of a flight route on TV or wherever, the route always looks like a gentle curve from one point to another, and not a straight line.  This is of course due to the curvature of the Earth, so when representing a truest direct path (upon the surface of the sphere, obviously, and not within it!) from two points separated by more than a degree or two on the surface, the line will begin to appear curved, thwarting that common sense adage about the shortest distance between two points being a straight line.  So with that in mind, trying to imagine how closely the various routes I plotted could create a true Great Circle route, was quite difficult.  On a 2D map, the images would appear showing my proposed route, but it would be very difficult to see if it actually matched a Great Circle in a spherical representation.  I know, not that difficult, for a thinking person, but let's not get pedantic shall we?  hehehe  Honestly I also just thought it would be incredibly fun to see the route in a 3D representation, because I have always been fascinated by higher dimensional visualizations than are typically available given the limited technology of my youth.  So...  I began to get an itch that there was a better, funner, way to go about the whole thing.

Now, I must digress at this point...  Around this time, my dear friend and colleague, a fellow escalation engineer at Microsoft, to whom I will refer simply as "HaidongH" purely for the purpose of fictionalization, was talking one day with me about a cool new feature he had seen in the new version of the .NET runtime, which our company had just released to much fanfare in our developer community.  The feature was called Windows Presentation Foundation (WPF), and was heralded for a number of reasons, but of primary interest to us, its ability to provide rapidly prototyped 3D graphics, for high level developers not possessing intimate knowledge of underlying Direct3D APIs, which are significantly more complex to use, and require a much higher degree of specialized knowledge to master.  Although a longtime avid fan of 3D graphics, and an extremely early adopter of ray tracing, with my very first computer, an Amiga 1000, back in 1985 or 86, it must have been?  (That is where I learned to program assembly for the first time, and was also exposed for the first time to 3D modelling, and ray tracing.)  So, when "HaidongH" told me about this new innovation, I of course excitedly fired up my Visual Studio instance, downloaded the beta of the new runtime, and kicked off a new project, to see what I could do!  And you may no doubt have realized now what my first project was bound to be!

So I began to write the 3D Earth Route Mapper...

Within a few days, thanks to the amazing simplicity and vision of the architects and implementers of WPF (one particularly helpful, to whom I will anonymously refer only as "sambent", and another I will anonymize as  "rrelyea", and yet another as "dwayneen", and one other I might as well through in, completely anonymously, as "erfabric"), I was able to achieve a remarkably lifelike implementation of a 3D Earth, complete with mouse controlled rotation in both planes, and zoom feature!  I was ecstatic, and could not help show my boss during our monthly check in and pow wow.  He was gracious to indulge me, and rather than remind me sternly that my productivity is not actually being enhanced too much by spending hours after work (...and let's just be honest shall we?  perhaps more than a few occasional moments stolen during work hours, even searching documentation sites using work owned assets!) wasting time on explorations completely unrelated to my actual duties as his direct report, warmly encouraged me, acted marveled (similarly, in my recollection, to how a parent must act when their little snowflake brings home that especially remarkable hand painting from craft station in kindergarten) and said not a word about my primary work duties, in regards to that segment of our conversation.  In my defense, I did assiduously maintain all of my duties at work, without fail, even during my most obsessive periods, when I occasionally failed to sleep a night or two, and worked the prior and subsequent days!  I did once call in sick after just such a night, but even then, my boss, who I must now anonymously name as "bhorn" never said one word, and accepted my honest explanation without any further inquiry.  So it is perhaps to him that I owe the greatest debt of gratitude, for this entirely superfluous and self indulgent branch that grew out of my life, which has nevertheless proved to be a source of much inspiration and delight, not only for myself, but for many friends and family, with whom I have been able to share it since.  Thank you then, once again, Mr. "Horn".

I must digress again once more at this point, to note, that Mr. "Horn"'s great foresight and vision were proven wise, and ultimately beneficial to his own higher ups, as was dramatically proven when a surprise emergency opening appeared months later, on the team supporting just those technologies I had endeavored to learn in my "vacant pursuit of my own self interest", which he so graciously allowed.  To add glory to grace, he further consented to allow me to be stolen from his team, to serve the other team for several years, a change which did put my former small team in a very tight circumstance for some time as well, but which he did for my benefit, and for the benefit of the greater enterprise we serve.  This proved to be one of the most happenstance advances in my career, and it was with great happiness that I returned to his team upon his request, with the situation stabilized for the user interface team to which I had been transferred during the intervening years.  Last week, in fact, I returned to his team, or to a newly formed incarnation thereof, which will include not only all of our prior areas of expertise, but also expand to include a significant body of other functionality represented in our company's enterprise database product, and its associated business intelligence offerings.

Although in the subsequent days, weeks, and ultimately months that followed, I did realize that a full actual implementation of my vision became significantly more complex "on the ground" than it had been previously in my mental model, my "map", of what the process would entail.  This has always proven to be my experience in the damned infuriating field of software development, but is no doubt also the source of the same drive that has brought me back to software again and again, on deeper and deeper levels, throughout my life's journey.  And so for its burden, I would never bear resentment, but can only feel joy upon my contemplation thereof.

Ultimately, I did develop a tool that met all my functional requirements, and was actually quite beautiful.  I ended up learning those lower level APIs that I had sought to easily avoid, when HaidongH first suggested the framework might be of interest.  And as I mentioned before, the whole exercise led to a wonderful career diversion into a whole range of lower level user interface APIs to which I had not previously been exposed.  I also gained valuable experience in dealing with threading and synchronization, one of the most subtle and technically difficult areas of programming to master.  Again, thank you Mr. "Horn" for granting me the freedom to achieve this within the parameters of a very flexible work environment.

The result of this magnificent waste of time was a tool, which did successfully allow me to plan and implement very approximately my goal to complete a true Great Circle.  Although the code is not production ready, and is not open source, I provide here some real images taken from my ultimate interface.  The primary adaptations introduced in this art which I claim as my own novel invention are an improved filtering functionality, and the 3D projection of a rotatable globe in order to visualize the filtered routes naturally.  The filtering features of the known prior art were extended, such that aircraft codes can be easily classified together, and a custom ComboBox style control, which allowed grouping of aircraft into their classes, such that one can simply select "747", and all of the numerous aircraft codes which comprise that class of aircraft will automatically be selected.  The 3D visualization of the data was extended to include some novel features as well, including hover based highlighting of all routes meeting any given filter criteria, from a given airport, to make browsing of potential destinations simple and efficient.  A further novel feature to the user interface was the addition of transparency controls, to allow the easy visualization of routes obscured partially or completely by the body of the planet itself, when viewing in 3D.

Here with all the airports and routes known in the database displayed, you can see that it becomes a morass of lines.  Adding a novel feature to allow highlighting of all routes out of any airport, based on the position of the mouse over the globe enables even this very dense visualization of data to make sense, to show here all the routes out of DFW on any airline or any aircraft type, highlighted, and engaging another novel feature, the controlable transparency of the globe itself, which allows us to visualize the routes' curvatures beyond the visible surface, which would otherwise be obscured by the body of the planet.

Here the same dataset, filtered down to only aircraft types of 747 and A30, dramatically reduces the available routes from which to choose!  The tooltip feature here is displaying the name and airport code of the highlighted airport, again DFW.
Demonstration of the novel feature implemented in my grouping ComboBox, which allows groups to be listed at the top of the entries, with a space between them and the actual individual items, upon which selection of the groups subsequently also select all their member items individually in the list as well.  You can see here the OneWorld alliance (where my airline miles reside) is selected, so the subsequent airline codes, 4M, AA, BA and so on, will all be selected automatically then from the list of airline codes below.

Demonstration of the grouping ComboBox in action.  Note the selection here of the 38- and 74- categories, for all A380 and 747 class aircraft, and their associated aircraft codes, sloppily highlighted.
And here you can see that if you scroll down, you do indeed find all the A380 aircraft codes selected subsequently

The zoom feature seen here can allow differentiation between route origins in a densely populated portion of the map, this showing all the OneWorld routes into London, where zoomed in, we can see that there are really three airports jammed into one very small region.  On the fully zoomed out globe, it would be impossible to differentiate and find the airport of interest.  With this feature, one can easily highlight an airport, even if it is close to a neighbor, which you prefer to ignore, so that you also exclude those routes only originating from the neighbor.

A demonstration of how we select only London's Heathrow airport, so we can see those routes, and were able to avoid seeing routes from Gatwick.

This one was just for fun.  Here I show all OneWorld routes, and highlighted in orange the route from Santiago, Chile to Sydney, Australia.  I was particularly struck by how different this route appears in the 3D visualization, compared to its appearance on a 2D map.  On this visualization, its curvature is clear, and you can see that it is wrapping around the "underside" of the planet.  This was a great example of how it was easier to visualize the 3D aspect using my superior visualization innovation for the same application as the known prior art, which I claim as part of this invention.
Finally, I present the route I ultimately took.  DFW-LAX-SYD-CNS-BNE-SYD-SIN-HKG-LHR-FCO-TLV-LON-MEX-DFW.  "But wait," I hear you ask, "That doesn't even really look like a Great Circle.  Wasn't that the whole point, to verify that your route approximated as closely as possible a true Great Circle route?  How did the tool even help in that regard then?!!" 
Here, we see the route, after rotating the sphere of the globe as much as possible to show it "head on", with transparency turned all the way down, so we can easily see the route whether obscured by Earth or not.  You can see that most of the route is barely visible in the image.  The reason is that it lies very closely along the exact edge of the sphere, from this angle, and therefore is nearly circular, and nearly congruent to the sphere, facts of which then nearly define what?  A Great Circle Route!  That is why we cannot see much of the route from this angle, viewed from above the deep southwestern pacific/arctic, and serves as evidence that we did indeed very closely approximate a true Great Circle route!

Note:  I must also thank my famous and brilliant fellow inventor, Charles Petzold, a library of whose was useful in saving me some more difficult calculations when doing some transformations for efficiencies sake, in which I actually draw my route lines as curves, on a 2D projection of the map of the earth, which are then correctly wrapped around a 3D sphere.  This is much more efficient computationally, and achieved a dramatic improvement, particularly with large numbers of routes displayed, since my initial effort was stymied by the realization that its computational intensity was sufficient to grind my rendering to a halt with more than a couple hundred simultaneously displayed routes.  Calculation of 2D projections for a flat display monitor of complex curves (actually, in 3D representation, there are not even any such things as curves, but rather, only triangle meshes, which must be strung together in the model of the space, to form things that only appear to be curves in 3D space).  So with his helpful guidance and some persistent experimentation, I was able to implement a significant optimization, requiring only a little math and a few additional special functions to achieve.  And therefore I must name his prior art as a dependent library upon which I built my own code, of the namespace Petzold.Media3D.

Finally, if you have actually endured my verbose enumeration of all the details of my superb and spectacular invention, please do read on, starting with my very first post on April 26, back in 2011, and celebrate with me the wonderful adventure, which was my most amazing life experience to date in a number of ways, to which I now refer affectionately as my "Around the World in 30 Days" trip!