Measuring GNSS accuracy on Android devices
How accurate is your location?
There are a lot of factors that go into choosing a new mobile phone. Typically, location accuracy isn’t the first thing that comes to mind. However, the accuracy of your device’s Global Navigation Satellite System (GNSS) receiver can have a huge impact on your mobile experience. Anyone who has tried to use real-time walking navigation or to find your Uber or Lyft ride in an urban environment can appreciate this as they’ve spun in circles chasing the blue dot on the map.
Let’s say you want to buy a phone that will give you a great location experience — how would you know which to pick? It’s not like device reviews typically discuss GNSS. And if they do, it’s usually a quote of the manufacturer specs, which looks like:
Satellite Systems Support: GPS, GLONASS, Beidou, Galileo, QZSS, Dual frequency GNSS
These specs look the same for most devices, with the exception of “dual-frequency”. The “dual-frequency” GNSS feature on newer Android devices has gotten a lot of attention lately, as using multiple GNSS frequencies is potentially a game-changer in terms of location accuracy — going from several meters of error (over 10 feet) to decimeter level (less than 1 foot).
So do these new dual-frequency GNSS devices have improved accuracy over their single-frequency counterparts? That’s a harder question to answer from specs alone— location accuracy depends not only on the GNSS chipset inside phone, but also the integrated antenna quality as well as the low-level OEM software processing the GNSS signals. GNSS signals themselves are very dynamic too — you’ll have a very different experience standing next to a skyscraper than I will inside a single-story wooden construction building.
Given the above, how can we tell determine the GNSS accuracy of a device? To solve this problem, let’s look at how most users and Android apps currently determine GNSS accuracy, the downsides to this approach, and how we can improve things going forward.
What is “estimated accuracy”?
When users want to find out more about GNSS on their device, they download utility apps such as GPSTest by barbeauDev (full disclosure, this is my open-source app), GPS Test by Chartcross Limited (yes, the similar names are confusing), or GPS Status & Toolbox by EclipSim.
Let’s fire these up and look at the data:
Each of them shows an “accuracy” or “error” value. That was easy — we’re done, right?
Well, no.
It turns out this value comes directly from the Android Location API, so it’s easy for developers to simply display this in an app, no questions asked.
To understand what this really means, we need to dig deeper into the Android developer documentation for Location.getAccuracy(), which says:
Get the estimated horizontal accuracy of this location, radial, in meters.
We define horizontal accuracy as the radius of 68% confidence. In other words, if you draw a circle centered at this location’s latitude and longitude, and with a radius equal to the accuracy, then there is a 68% probability that the true location is inside the circle.
It may sound foreign, but you’ve seen this value before — it’s the radius of the blue transparent circle surrounding your position in Google Maps and other apps:
The word “estimated” is really important (my emphasis), and here’s a visual representation:
There are two immediate takeaways from this graphic:
- The estimated accuracy (thick black line) isn’t equal to true accuracy (red dashed lines)
- By design, about 32% of the time the true accuracy (error) is greater than the estimated accuracy — in other words, the device is underestimating the error
Another important note is that the spec that defines the 68% confidence requirement is written by Google as guidance for Android device manufacturers such as Samsung and LG. However, the device manufacturers create the hardware and lower-level software that actually calculate this estimate accuracy value using dilution of precision and other GNSS data. To my knowledge, this data from each device isn’t reviewed by Google or anyone else prior to device launch. And past research has shown that some device manufacturers haven’t met the 68% threshold requirement.
In summary — you wouldn’t trust a device review written by the manufacturer, so why would you trust an estimated accuracy value from the same OEM?
A better way to measure accuracy
So how do we determine the true accuracy, or error, of GNSS in an Android device?
Conceptually, it’s pretty straight-forward — compare each calculated position to your known actual location. And that’s exactly what the new “Accuracy” feature in GPSTest allows you to do:
You can tap on the map or type in your actual location information (shown on left), and then GPSTest will measure the difference between this “ground truth location” and the locations calculated by your device GNSS (“Error” and “Avg Error” shown on bottom of right screenshot).
If you drag the bottom panel up, it will show you a graph with red lines for horizontal error and, if you’ve entered altitude information, vertical error:
The “estimated horizontal accuracy” also appears on the graph as the blue line — so you can compare how accurate your device thinks it to how accurate it actually is. You can also see the “estimated horizontal accuracy” as the traditional blue transparent circle on the map for the current location.
In the left screenshot, you can see that this device (a Samsung Galaxy S8+) is doing a decent job of estimating the error at the beginning of the test— the estimated accuracy is the same as the actual error for the most recent position. In the right screenshot, though, you can see that as the test progressed the device started consistently underestimating both the horizontal and vertical error (the blue line is less than the red line).
Tips, tricks, and caveats
Here are some things to keep in mind when using the “Accuracy” feature of GPSTest:
- Don’t move during the test — your device should remain completely stationary during static accuracy tests for optimal results (See this paper for a methodology for “dynamic” accuracy when the device is moving — maybe a future feature in GPSTest?)
- Choose your ground truth location carefully — There can be errors in aerial imagery that affect results if you choose your location by tapping on the map. Ideally, you’d go to a surveyed location marker and manually enter the latitude and longitude for that marker (converting to the WGS84 datum first if needed). If you’re in the U.S., see the National Geodetic Survey Survey Marks and Datasheets page for more details on finding a survey marker near you.
- Altitude must be entered in meters above the WGS84 ellipsoid, NOT meters above mean sea level. Here is a good article describing the difference between these values. See this issue for more details.
- Horizontal error is measured using the Android
Location.distanceTo()function. If you look at the Android source code, this function uses the Vincenty Inverse Formula to calculate the distance between two points (observed and ground truth locations) on the surface of the WGS84 ellipsoid. Vertical error is the simple difference between the observed and ground truth altitude values, and can be positive or negative. - “Estimated vertical accuracy” is only available on Android 8.0 and higher. The exact definition is available in the
Location.getVerticalAccuracyMeters()documentation.
Conclusions
Don’t trust the estimated accuracy value you see in GNSS apps. If you’re not entering your ground truth location into an app, you’re seeing estimated accuracy, not actual accuracy.
Give the new “Accuracy” feature in GPSTest a try and tell me what you see in the comments below, especially if you have a dual-frequency GNSS device!
Was this article helpful? Consider following me on Medium. If you’re a user of the open-source GPSTest app and you’d like to support it, you can check out the GPSTest “Buy me a coffee” page:
