Milermeter, Google and gmaps (LOL)
Hello all, I bet you’ve been noticing a ton of changes on the site lately. I’m sorry I haven’t let you know what was coming in advance, but for reasons I’ll explain I’ve been too busy making the changes happen under a deadline, and now I have time to explain.
As of July 15, google announced drastic pricing changes to the cost structure of google maps. (Here’s a pretty good article about it.) Without getting too deep into the weeds, it meant that a number of services that once cost me a grand total of $0.00 became so expensive that I would have been losing money (a lot) on the site. (They gave a two month credit to get people used to it, so my real deadline was September 15.) I had to scramble to make changes, because though this site is a labor of love, I’m in no position to suport something that would lose money each month. (It’s really more of a labor of love with benefits.)
This has meant a number of changes, which I’ll outline below. I think I’ve been able to get just as good results as before (sometimes better!) Note that in every case, I haven’t been able to replace formerly free google services with other free services; I’m still paying much more than used to, just less than google’s exorbitant rates. I’ve learned a lot about the current landscape of google map competitors in the process, and it’s amazing to see how the field has grown. In fairness, google created this category, and even subsidized it to a large extent (little did I know how much,) so in a small way I’m still grateful to them. Though mainly right now I think they’re greedy and kinda hate ’em. Here’s the list of changes.
You’ve no doubt noticed that the map layers themselves have changed. I ended up changing this twice. First I changed from google to mapbox (a mapping-only competitor that offers basically all the same features, using OpenStreetMap data.) It turned out that mapbox’s pricing, while better than google’s, wasn’t as great as I’d thought, and so I had to switch off that too. (This article is a pretty good explanation) In the end, I bought a yearly license to use data from https://openmaptiles.com/ and am now hosting the tileserver myself. Honestly? From a nerdy-learning-opportunity perspective I really enjoyed setting up the tileserver, as it gave me a chance to run docker in production, set up nginx as a reverse proxy, use letsencrypt for free SSL, and hone my AWS skills.
You’ve probably noticed that the maps load differently now. They’ll seem to build themselves and fill in details as you zoom in, where they used to start out as blurry blocks and get get less progressively less blurry as data loaded in. This is because the switch gave me the opportunity to use vector tiles instead of raster — basically my tileserver sends you a document that tells your browser how to draw the map itself, instead of sending a batch of images to make up the map. It should make for quicker and smaller downloads of the maps (though it’s true it puts more of the work into your browser.) I happen to really like the effect, and I hope you do too.
This has also meant something of a brain transplant for the code. All the API calls that formerly called the google maps API now call similar code provided by mapbox, and this change was the lion’s share of the work I had to do. It means a lot has changed under the covers, so if you find a bug, that could be why (please let me know if you do.) That was why, for a little while there, the ability to edit routes in the middle (rather than just adding/removing points from the end) was disabled. It was the most time-consuming retrofit, and I just had to push it out til later to stay under the deadline imposed by google for pricing changes.
It turns out that sending a giant super high resolution camera into space on the nose of a rocket, and taking pictures of Earth with it for decades, is kind of expensive. I now know that this is one of the biggest costs that google was subsidizing. The provider I finally landed on for my map tiles had satellite data, but at much lower resolution, to the point of being basically unusable. I stayed with mapbox, who had better images, but it was still costing 40-50 dollars per day and wasn’t something I could continue to pay for. So, for a while, the site was set up to seriously curtail the amount of satellite imagery you could get. You could do a “glance” of satellite or hybrid images, then it would flip back to a map whenever you moved or zoomed.
Some of you guys reeeeeeeaaaaaallllly hated this.
I don’t blame you. I thought it sucked too. But some of y’all should really work on being a bit more polite about the emails you send to strangers on the internet.
Anyway. There’s good news. There’s a company called ESRI who sell all kinds of geographical data. They do so much business, in so many different types of data, it was a little hard to get a straight answer from them about whether I could get satellite images to use on website. It turns out I can! I literally had to talk to three different people in sales before I got to the right one who understood the type of license I need, but now the site has satellite and hybrid data again! Looks to me like the resolution isn’t quite what google had on offer, but I hope you’ll agree it’s enough to get the job done. If you’re curious, ESRI is charging me $500/year for this data, so it’s not like they’re giving it away, but it’s a rate that’s much easier to support. Interesting nerd note: the satellite images come from ESRI, but road outlines for the hybrid version are now coming from the tileserver I set up! I think that’s kinda cool.
This is another thing I was getting from google, and moved to a different provider for. Many of you noticed this and reported that the data quality was just not the same — whole countries no longer being found, and the results just not as consistent. I did some number crunching, and this is one of the (relatively) less significant google costs, around $10/$15 a day. I made the call that the quality of the data we were getting from google was worth it. And yes, out of bitterness, I would rather not give them the money each day, but I do care about the quality of the features, so google, you win this round.
One of the important features of the site is the ability to draw the route between the points you clicked. This is another thing that google was going to start charging for, that had been free before. It turns out there is an open source project to offer this feature based on Openstreetmap data, and I was able to stand up a server in AWS using this code to replace google’s version of the service. It’s quite cool how it works. It maintains a graph representing the street network of the whole world in RAM so the results come back fast. Which also means that it requires a very beefy server with loads of RAM, which is pretty expensive to run. That and the geocoding are the two biggest costs I had to take on in the course of this project. Long story short, there was a change under the covers to this feature, but hopefully you won’t notice a difference.
I wasn’t too thrilled with how I got treated by google after being one of their very first generation of mapping mashups. In case you discovered the site later, it was launched in 2005, (when people actually used the word “mashups”) and it was literally on the google maps API from the day it came out. But in the end, the work I’ve had to do to handle this change has been technically interesting and a great learning experience. And to be fair, many of the google competitors I’m now using almost certainly wouldn’t exist (or not in the same way) if google hadn’t disrupted people’s relationship with maps and geography. Now maybe google will be disrupted (I hope!) by other upstarts, which is as it should be.
I’m pretty much stuck with the domain gmap-pedometer.com for the site due to all the links and SEO out in the world that use it. It was always a bad domain name (too long, and what the hell does it mean?) and now there’s no “gmaps” left in the app at all. Serves ’em right!
As it turns out, I had a giant vulnerability in this business in the form of vendor lock-in — my fate was entirely determined by google. Not anymore! I intend to still maintain milermeter.com for the foreseeable future, and I wish you many many refreshing runs, walks, bike rides, and of course, pleasant hours seeing how far you went on the site.
Please keep coming back, apologies for the recent bugs and disruptions, and thanks for your patronage.Read more
UPDATE 9/18/18: Thanks to everyone who had reported that the recent changes broke this again, I have updated this post to fix the bookmarklet. You will need to get a new copy of the bookmark from one of the links below, the old one will not work correctly.
GmapToGPX is a bookmarklet that makes it possible to export your milermeter.com/gmap-pedometer.com routes as GPX tracks.
- Drag the following bookmarklet GmapToGPX to your browser toolbar
- Open a route on gmap-pedometer.com, here’s an example
- Wait for the map to load, then click the bookmarklet you dragged to the browser toolbar
- You’ll get a window with the GPX context, select all, copy and paste to a file (named .gpx or .txt) and load into your favorite GPX tool
- If you need to convert a GPX to a Garmin TCX, you can use www.gpsies.com
Thanks to the users who have pointed out that the GmapToGPX bookmarklet has stopped working. To fix this issue, delete your current bookmarklet and instead drag this one: GmapToGPX to your bookmarks bar.
The issue with the previous bookmarklet was simply that it referenced a URL in plain HTTP, and the milermeter.com site has been converted to use HTTPS for security,
Note that this utility had previously been kindly developed by a third party developer, and it seems this issue has been brought to his attention, but hasn’t been addressed. I’ll have this page up until further notice
UPDATE: I just created this video showing the process on Mac/Firefox: enjoy!
I ate five salads a day as part of my fitness challenge. You won’t believe what happened next.
Week one: At first I am skeptical of the five salads a day thing, but I decide to give it a chance. I cry when I throw away all the honey mustard in the fridge. But within days, my persistent cough starts to go away: I am spending so much time chopping salad vegetables I don’t have time to smoke. With the money I save on cigarettes, I buy a majority stake in Snapchat.
Week two: Once I switch to eating only locally-sourced produce, my senses begin to sharpen. Out in the yard, I smell the freshly-mown grass and regain my childlike sense of wonder and innocence. Unfortunately, I can hear the neighbor’s kid from down the street playing Hot Cross Buns on the recorder so many times in a row it sounds like he is rehearsing for a Phillip Glass opera.
Week three: I develop superpowers. I realize that, under the yellow sun of Earth, I can do things normal humans cannot: run a mile in two minutes, lift a car bare-handed and log out of Facebook. I also score an eight-letter word in Scrabble!
Week four: I leave my material body behind and become a being of pure consciousness. I am plugged directly into the universe and have access to all frequencies of energy in all times. Bonus: My complexion has totally cleared up and I fit into my jeans from college again.
First Time Triathlete Part 2: Swimming
I’m a great swimmer. Always have been. Feels completely natural and easy for me. It was a fulfilling and positive experience last time I put in serious laps in the pool. 25 years ago.
I’m pretty proud of myself that I’ve made a point of getting some of the basic training for my triathlon done before the start of the official 8-week training program. Instead of my usual daily run, this week has been a run, two bike rides, and a swim.
Biking is almost as easy to kick off as a run. For a run, strap on your shoes and go. Additional equipment and ahem, planning and record-keeping, are fun but optional. Biking is almost the same, but with a bit more equipment. Bike, helmet, titanium ass implants (oh, how I wish.)
Swimming requires a bit more planning. A place to swim. Goggles. A few different necessaries that took me until late in week 0 to get together. My race will have a 1.5k swim. Turns out my pool will be 25 yards. A little fun with math later, and it’s clear I’ll need to work my way up to 66 laps. My swim fitness class a little while ago (you know, college YES IT WAS 25 YEARS WHAT’S YOUR POINT) culminated in a swim just about that length. No sweat.
Ok, in hindsight, it’s really obvious where this is going.
Swimming is really really different from running. And honestly, I have no idea what it felt like when I was a beginning runner 12 years ago. I’ve completely forgotten. That memory is buried under literally thousands of miles of pounded pavement. And all those miles came after the last time I swam laps. Which means I probably was trying to swim my very first laps in 2.5 decades like I run now, at a pretty agressive pace, maybe 75% effort.
Protip: don’t do that.
So, it’s humbling to need to stop and gasp and cling to the wall after 5 laps. There’s a learning experience I can chalk up from this training program.
And here’s another: fear. You know what happens when you get really really tired running? You stop, walk a while. Depending on the size of your ego, it’s sorta embarrassing. Maybe you can manage to do it on a pretty quiet part of the course.
You know what happens when you want to stop while you’re swimming? You swim anyway. Because if you don’t, you drown.
And you know how my race will be different from the training I managed to conduct for 125 yards at my best stretch this week? No wall. Open water. Nowhere to gasp and rest and steel my resolve.
I’m not giving up yet. That was my first swim in 25 years. (Crap, I really never thought about how long it had been.) I’m pretty sure I just got my pace very very wrong, and I can fix this by starting out much slower next time. I stole a few moments at the pool with the kids this afternoon, took it slow and got to 6 laps pretty easy, still felt like I had plenty in the tank.
But this will be more of a stretch than I thought.Read more
First Time Triathlete: Part 1
I, as many runners like me, finally reached a certain point. A point where voluntarily running 26.2 miles wasn’t… quite… stoopid… enough.
There are many solutions to this. Some will dream up a costume or run in a full military or civilian service uniform. Those inclined toward becoming being highly caffeinated mountain goats will try ultras. Those who want to trade mind-erasing endurance for speed go for a 5k.
Me, I thought I’d attempt a triathlon. It’s not as bad as you think; I have no illusions about my ability to complete a full Ironman. (MUCH respect to all who have.) Fortunately for me there are many triathlon distances to choose from; I’ll be trying an Olympic length, the Westchester Triathlon in my own home town of Rye NY. Easy, yes? 10k run, 1.5k swim, 40k mile bike. Only I don’t ever, you know, swim or bike.
I have a relationship with this race already. It’s in my own town of Rye NY, which means that I have, for a few years now, been in the middle of a late September Sunday long run, training for one marathon or another, when it was being held on the quiet roads of the town of Rye. It’s possible that I may have mooched off its water stops a time or two.
Marathons are still a challenge. But a familiar one. I’m looking forward to being a raw newbie at this. There are bound to be a few massive mistakes along the way. (Will the first one be thinking I can be ready for this in 8 weeks?? Whee!) Or not. I surely underestimate my studliness (shut up, you.) Either way, I’ll record them here. Hope you enjoy.Read more