Accuracy problems

Hi,
I posted this on a group on facebook, and was told to post it here aswell, so you guys could have a look at it.

Hi,
I’m facing a accuracy problem with DroneDeploy and elevation. We have mapped 18 holes of a golf course, and stitching it together, each hole separately, and 1 to 9 and 10 to 18. When stitching each hole, separately, the accuracy is ok (not good, but ok-, can be way off for all i know), but when applying all of the same images (1-9), the accuracy drops way off (22-35ft on elevation, rivers running upstream ect).
We also tested using Photoscanner to do the job, and here we dont have the same problem. (The elevation make sense)
I know we should use GCP to get it better, but i still dont understand why it get so much worse, when using the same images.

We are flying a DJI Inspire 2, using an IPad mini with the latest OS, and used Maps Made Easy’s app

Thanks

Are you guys taking off from the same spot each time?

No, but Photoscanner had no problem with that.

Hi @Droneservices,

Each map uses the takeoff location of the drone as a “zero” value. This can vary, depending on the accuracy of the GPS, by a bit so it won’t always be exactly zero if you drop a pin on the map. When you combine multiple maps the elevation throughout the map will be averaged throughout the map. This is why @markaguille asked if you used the same takeoff location for each of the sections of the golf course. By doing that you can mitigate any large discrepancies when elevations are averaged.

By using GCPs you could tie these points to precise RTK GPS which would solve this issue. How did you merge all of the maps together within DroneDeploy?

Best,
Zach

Hi @zach1.
Thanks for feedback. We did not merge the maps together, but made 2 bigger maps with all of the images. (around 650-700 on each big map)
We flew each golf hole individually (some we flew 2-3 holes in one go), than made a map for each hole (1,2,3…) and than 2 bigger maps with hole 1-9 and 10-18.

This is the accuarcy on one of the bigger maps

RMSE 8.9m X 1m Y 5.3m Z 7m

http://www.dronedeploy.com/app2/data/59ad422b2cee5022dbd2646b;jwt_token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ0eXBlIjoiUmVhZE9ubHlQbGFuIiwiaWQiOiI1OWFkNDIyYjJjZWU1MDIyZGJkMjY0NmIiLCJleHAiOjI1MzQwMjMwMDc5OX0.GsnA0eAGlpYUhmjxxSDZ72A077OTI2rp6izQTAp_1vtRjsuvQxjcg3iKFkibyuaLtcKa0e1B_6-OkQo0bRkK0g

Hi @Droneservices,

I looked at your accuracy (RMSE) for your smaller and larger maps which I have listed below.

Front 9 - 18.2ft
Back 9 - 29.1ft
Hull 8 - 20.5ft
Hull 18 - 18.4ft
Hull 9 - 17.8ft
Hull 1 - 14.8ft
Hull 6 - 14.9ft

From the two large maps, the RMSE ranges between 18ft - 29ft, and for the individual holes the RMSE ranges between 15ft - 19ft. The elevation (Z) for the individual holes ranges between 1.1ft to 11.2ft.
RMSE relies solely on the accuracy of the GPS on your drone where many conditions factor in. Some of those factors include GPS strength, calibration of the GPS, type of GPS, same location for take-off and landing. If the take-off and landing locations are different, this will create room for error.
The accuracy we are seeing above is normal for a standard GPS onboard an Inspire 2. As Zach mentioned in the post earlier, GCP points will get you accuracy within inches.

Did you fly all these missions on the golf course with DroneDeploy?

Thank you,
Nipul

Hi Nipul
Thanks for answar.
We had several takeoff loacations. We used Maps made easy for the missions, and had terrain awareness on, so that could also cause problems, have i learned now!
Seems like we need to go back, and add GCP to the mission.

Jørgen

Hi @Droneservices,

Due to the inaccuracies, we definitely recommend using DroneDeploy to fly your mission. Once you add GCPs to your mission, the accuracy of your Z levels will shoot up.

Nipul

If you use DroneDeploy with a Phantom 4 Pro drone, you can get cm-level elevation accuracy without GCP’s for well textured surfaces on your maps flown with 80%/80% overlaps on overcast days with a flight pattern that includes the take off location. Here’s how:

Fly the mission with DD, upload the pictures and then look at the 2D map. Use the Measurement Annotations to measure the location of the takeoff point. Look at the elevation reported. Generate an Offset = Actual Elevation - Elevation Measured. Add this offset to the AbsoluteAltitude entry in the Exif data of all the pictures for this mission and reprocess the pictures to generate a new map. Now when you look at measured elevations, the takeoff location will show the actual elevation and all other elevations will be relative to this.

The reason this works for the P4P is that all its altitude data stored in the Exif data of your pictures is based upon the on-board barometer. This is a ying and yang thing. On the one hand, barometer readings typically vary slowly during a 10 minute mission helping the P4P fly level resulting in small variations being recorded in the RelativeAltitude and AbsoluteAltitude field of the Exif data. DroneDeploy, which uses the RelativeAltitude, thus can generate maps with cm-level relative accuracy under good flying conditions. But on the other hand the absolute elevation accuracy can be awful for missions flown on different days since the barometer-base altitude recorded in the AbsoluteAltitude field of the Exif data (and used by DroneDeploy to print out an elevation) varies by 100’ for each 0.1" Hg change in the barometric pressure (which typically ranges over 1" in the course of a year and 0.5" in a couple of weeks). This potential variation in the elevation makes it hard to compare results from missions not taken on the same day or, when the barometric pressure is changing rapidly, within a 15 minute period. Adding the offset removes this source of variation from the elevation data reported by DD.

I have used the procedure described above to generate a map which appears to have cm-level absolute elevatuon accuracy on the hard surfaces of my map that are well textured. I have done several measurements to confirm this by hand using a laser level but it is a slow process since I do not have survey-grade equipment to facilitate measurements over a large area.

But your case is more complicated since you takeoff from different elevations. It would seem, based upon my apparently successful procedure above, that you could follow a similar procedure and accommodate the different elevations by adding, separately for each mission, to the RelativeAltitude field an OffsetR = Takeoff Elevation - Elevation of Reference and adding to the AbsoluteAltitude field an OffsetA = Actual Elevation - Elevation Measured + OffsetR. Pick one mission to be the Reference and adjust the photos for each other mission based upon this.

I have not yet tried this but it may work. I use a simple 20-line Python script to change the AbsoluteAltitude field in the Exif field of all photos in one mission. It takes less than a minute. The same script could be used to change the RelativeAltitude field.

If you do not fly a P4P or other drone that records barometer-based altitude measurements or you are not into scripting (I would be happy to send you my script), then adding GCP’s may be your best approach.

Regards,
Terry.

Hi Terry,
I am interested in trying this script of yours, any chance of you sharing it with me?

@markaguille,

It would great to have you give it a try. Below is a copy of what I wrote in the Steep Terrain Question topic on the DD Forum. This may be of help and shows a copy of my script.

You can do this job without GCPs since you are mainly concerned with adjusting the large offset in the elevation. To do this, you can use an Auto-Calibration Technique to achieve cm-level accuracy in the elevation data. This is done by first making maps using only 30 photos from a mission. Choose photos that encompass a point with known elevation, say where you would have placed a GCP. Process this subset of photos into maps. This will go very quickly. Then on the 2D map, measure the elevation of the known point. Now compute an offset = Known elevation of point - Measured elevation of known point. Next add this offset to the AbsoluteAltitude field in all the photos of the mission and generate new maps using all the photos. The new maps can have cm-level elevation accuracy if the elevation of the known point has cm-level accuracy and the known point is well modeled in DroneDeploy, which will happen if you put a high-contrast GCP target at the known point.

If you do this for each of your missions (or all the missions you ever fly so that you always have excellent elevation data in your maps), then you can combine photos from multiple missions to make one big map with accurate elevations (at least as accurate a DroneDeploy will give you, which as I am sure you know, is a function of many things, especially surface texture).

GCPs are better if you want better GPS longitude and latitude data in your maps also. But for your case, the elevation error due to the offset in launch elevations is 10 to 100 times larger. So you get the biggest bang for your buck by just correcting the elevation data at zero cost.

The only technical part of this approach is adding the offset to the AbsoluteAltitude field in the Exif data of the photos. For that I wrote a Python script that does a binary search and replace of the AbsoluteAltitude field. Perhaps you can also do this or have a friend that can do it for you. The key is to change the AbsoluteAltitude field that looks like this:
drone-dji:AbsoluteAltitude="+275.96"
to this when, for example, the offset is 11.04 meters:
drone-dji:AbsoluteAltitude="+287.00"

Anyway this is working great for my missions and, since I am on a free Pro trial with no support of GCPs, it was something I could try out. I also learned that I could not manually edit the Exif data using a simple editor like Notepad in Windows 10 since it mangles the Exif data after saving. Nor could I use the widely available exiftool by Phil Harvey since it writes out the data in a format that DroneDeploy cannot parse. My 20-line Python script automatically processes all the photos from a mission to add the offset, taking about 100 sec to process 143 photos with 1 GB of data. My script is not very portable because it runs inside the 3D Rhino program using their RhinoPython variant but it can easily be ported to another Python version or other language. A copy is below. I had to use __ to show leading spaces as otherwise they are squeezed out of the post and note that the difficult to decipher ‘’’’ is actually ’ followed by " followed by '.

Python program to add offset to DJI AbsoluteAltitude field in Exif data of all the photos in a directory.
from time import time
import System
startime = time()
altitudeOffset = 11.01 # Set this to: Offset = Actual altitude - altitude measured on DD map
dir = ‘C:\Users\Terry\Pictures\fixelev’ # Set this to directory with photos
key = ‘drone-dji:AbsoluteAltitude="+’ # Fixed part of search pattern
files = System.IO.Directory.GetFiles(dir, ‘*.JPG’)
for file in files: # Offset the altitude in all photos
__ f = open(file,‘rb’); s = f.read(); f.close() # Open, binary read file and close
__ i = s.rfind(bytes(key)) + len(key) # Find pointer to value of absolute altitude
__ if i == len(key)-1: # then rfind returned -1 since it did not find key
____ print ‘The key:’, key , ’ was not found in file ‘, file, ‘\nPlease fix and retry’
____ exit()
__ oldAlt = str(s[i:i+10]).split(’"’,1)[0] # 10 works for up 100,000ft; Remove " and beyond
__ original = bytes(key + oldAlt + ‘"’) # Construct byte pattern for search
__ new = float(oldAlt) + altitudeOffset # Add offset to old altitude
__ fixed = bytes(key + ‘{0:.2f}’.format(new) + ‘"’) # Construct replacement bytes
__ s=s.replace(original, fixed, 1) # Search and replace bytes
___ with open(file, ‘wb’) as output_file: # Open file for binary write.
_____ output_file.write(s) # Write byte array to file.
__ print file, oldAlt, ‘{0:.2f}’.format(new) # Write summary of change for each file
print ‘Time = {0:.3f} sec to add altitude offset of {1:.3f} to {2} files.’.format(time()-startime, altitudeOffset, len(files))

This Auto-Calibration Technique is like creating a virtual, altitude-only GCP which is all you really need in some situations. And it only takes about 30 minutes of preprocessing time to generate the 30-photo map and update the altitude data. Plus it can be applied retroactively, allowing you to improve the elevation accuracy of old maps. You just need to have kept the photos and be able to identify a point with known elevation that has enough texture to be well modeled by DroneDeploy. I do nothing special for my missions, using no special targets. I just use a concrete surface that I know is at 1000’ elevation. This surface has an exposed-aggregate finish which is enough for DroneDeploy to model well in 3D.

I recommend doing this for all your missions whether or not you are combining pictures from multiple missions because it always gives you maps with accurate elevation data with zero added cost. Otherwise the elevation can move around by up to 1000’ over the course of a year due to the typical 1" variation in barometric pressure. I know this is true for my Phantom 4 Pro because it uses its on-board barometer to calculate the altitude. Over a 2-week period, the measured elevation of the same point on maps from different missions varied by 500’. And I suspect most drones will see this same issue as they likely use their on-board barometer to calculate the altitude that is recorded in the Exif data and to fly level during a mission.

Regards,
Terry.