gmap-pedometer.com blog

Hello all, I bet you’ve been noticing a ton of changes on the site lately. I’m sorry I haven’t let you know what was coming in advance, but for reasons I’ll explain I’ve been too busy making the changes happen under a deadline, and now I have time to explain.

As of July 15, google announced drastic pricing changes to the cost structure of google maps. (Here’s a pretty good article about it.) Without getting too deep into the weeds, it meant that a number of services that once cost me a grand total of $0.00 became so expensive that I would have been losing money (a lot) on the site. (They gave a two month credit to get people used to it, so my real deadline was September 15.) I had to scramble to make changes, because though this site is a labor of love, I’m in no position to suport something that would lose money each month. (It’s really more of a labor of love with benefits.)

This has meant a number of changes, which I’ll outline below. I think I’ve been able to get just as good results as before (sometimes better!) Note that in every case, I haven’t been able to replace formerly free google services with other free services; I’m still paying much more than used to, just less than google’s exorbitant rates. I’ve learned a lot about the current landscape of google map competitors in the process, and it’s amazing to see how the field has grown. In fairness, google created this category, and even subsidized it to a large extent (little did I know how much,) so in a way I’m still grateful to them. Though mainly right now I’m pretty annoyed with them. Here’s the list of changes.

Maps

You’ve no doubt noticed that the map layers themselves have changed. I ended up changing this twice. First I changed from google to mapbox (a mapping-only competitor that offers basically all the same features, using OpenStreetMap data.) It turned out that mapbox’s pricing, while better than google’s, wasn’t as great as I’d thought, and so I had to switch off that too. (This article is a pretty good explanation) In the end, I bought a yearly license to use data from https://openmaptiles.com/ and am now hosting the tileserver myself. Honestly? From a nerdy-learning-opportunity perspective I really enjoyed setting up the tileserver, as it gave me a chance to run docker in production, set up nginx as a reverse proxy, use letsencrypt for free SSL, and hone my AWS skills.

You’ve probably noticed that the maps load differently now. They’ll seem to build themselves and fill in details as you zoom in, where they used to start out as blurry blocks and get get less progressively less blurry as data loaded in. This is because the switch gave me the opportunity to use vector tiles instead of raster — basically my tileserver sends you a document that tells your browser how to draw the map itself, instead of sending a batch of images to make up the map. It should make for quicker and smaller downloads of the maps (though it’s true it puts more of the work into your browser.) I happen to really like the effect, and I hope you do too.

This has also meant something of a brain transplant for the code. All the API calls that formerly called the google maps API now call similar code provided by mapbox, and this change was the lion’s share of the work I had to do. It means a lot has changed under the covers, so if you find a bug, that could be why (please let me know if you do.) That was why, for a little while there, the ability to edit routes in the middle (rather than just adding/removing points from the end) was disabled. It was the most time-consuming retrofit, and I just had to push it out til later to stay under the deadline imposed by google for pricing changes.

Satellite

It turns out that sending a giant super high resolution camera into space on the nose of a rocket, and taking pictures of Earth with it for decades, is kind of expensive. I now know that this is one of the biggest costs that google was subsidizing. Some companies already provide a similar service to their clients as they help provide high-resolution historical satellite images and many other things.  The provider I finally landed on for my map tiles had satellite data, but at much lower resolution, to the point of being basically unusable. I stayed with mapbox, who had better images, but it was still costing 40-50 dollars per day and wasn’t something I could continue to pay for. So, for a while, the site was set up to seriously curtail the amount of satellite imagery you could get. You could do a “glance” of satellite or hybrid images, then it would flip back to a map whenever you moved or zoomed.

Some of you guys reeeeeeeaaaaaallllly hated this.

I don’t blame you. I thought it sucked too. But some of y’all should really work on being a bit more polite about the emails you send to strangers on the internet.

Anyway. There’s good news. There’s a company called ESRI who sell all kinds of geographical data. They do so much business, in so many different types of data, it was a little hard to get a straight answer from them about whether I could get satellite images to use on website. It turns out I can! I literally had to talk to three different people in sales before I got to the right one who understood the type of license I need, but now the site has satellite and hybrid data again! Looks to me like the resolution isn’t quite what google had on offer, but I hope you’ll agree it’s enough to get the job done. If you’re curious, ESRI is charging me $500/year for this data, so it’s not like they’re giving it away, but it’s a rate that’s much easier to support. Interesting nerd note: the satellite images come from ESRI, but road outlines for the hybrid version are now coming from the tileserver I set up! I think that’s kinda cool.

Geocoding

This is another thing I was getting from google, and moved to a different provider for. Many of you noticed this and reported that the data quality was just not the same — whole countries no longer being found, and the results just not as consistent. I did some number crunching, and this is one of the (relatively) less significant google costs, around $10/$15 a day. I made the call that the quality of the data we were getting from google was worth it. And yes, out of bitterness, I would rather not give them the money each day, but I do care about the quality of the features, so google, you win this round.

Routing

One of the important features of the site is the ability to draw the route between the points you clicked. This is another thing that google was going to start charging for, that had been free before. It turns out there is an open source project to offer this feature based on Openstreetmap data, and I was able to stand up a server in AWS using this code to replace google’s version of the service. It’s quite cool how it works. It maintains a graph representing the street network of the whole world in RAM so the results come back fast. Which also means that it requires a very beefy server with loads of RAM, which is pretty expensive to run. That and the geocoding are the two biggest costs I had to take on in the course of this project. Long story short, there was a change under the covers to this feature, but hopefully you won’t notice a difference.

Conclusion

I wasn’t too thrilled with how I got treated by google after being one of their very first generation of mapping mashups. In case you discovered the site later, it was launched in 2005, (when people actually used the word “mashups”) and it was literally on the google maps API from the day it came out. But in the end, the work I’ve had to do to handle this change has been technically interesting and a great learning experience. And to be fair, many of the google competitors I’m now using almost certainly wouldn’t exist (or not in the same way) if google hadn’t disrupted people’s relationship with maps and geography. Now maybe google will be disrupted (I hope!) by other upstarts, which is as it should be.

I’m pretty much stuck with the domain gmap-pedometer.com for the site due to all the links and SEO out in the world that use it. It was always a bad domain name (too long, and what the hell does it mean?) and now there’s no “gmaps” left in the app at all. Serves ’em right!

As it turns out, I had a giant vulnerability in this business in the form of vendor lock-in — my fate was entirely determined by google. Not anymore! I intend to still maintain milermeter.com for the foreseeable future, and I wish you many many refreshing runs, walks, bike rides, and of course, pleasant hours seeing how far you went on the site.

Please keep coming back, apologies for the recent bugs and disruptions, and thanks for your patronage.