In making viewshed calculations, the earth's curve and the refraction of light in air have an impact (albeit minor) on lines of sight. I this post, I provide some links describing how Arc calculates both variables. I would love to find out how to recalculate the "refractivity coefficient" of 0.13 to other situations. I also put a plug in for fuzzy viewsheds, a drastic improvement that was first suggested 15 years ago, to be mostly ignored in practice until very recently.

Corrections for the Curvature of the Earth

ArcGIS 9.2 makes it much easier to correct for these two variables with standard corrections, but it is not clear how it makes these corrections, or how they might be modified. The help file is clear on how to modify the parameters of the analysis. At the bottom of this page, they explain calculations for the curvature of the earth.

How Viewshed Works

Another help file focuses on how curvature is calculated:

How Curvature Works

A more detailed technical article from ESRI elaborates on this:

Problem: Are there any more details on the outputs and calculations of the CURVATURE function?

A very useful post (by someone who usually puts up great posts) on the ESRI support forums explains this as well, as he describes how to calculate curvature manually. This was before the days of Arc's default correction, which seems perfectly good for estimating a variable that usually has minor effects (objects 10 km away look about 7.8 m higher; however, the effects are quadratic, so at longer distances, this is a big deal).

I used a favor to find out from the horse's mouth how Arc 9.2 makes these calculations, and some unnamed ESRI folk passed on that both the curvature and refraction variables are based on this article (attached to this post):

Yoeli, Pinhas, 1985. The Making of Intervisibility Maps with Computer and Plotter. Cartographica 22:88-103.

The Refraction of Light in Air

Mostly satisfied with the modeling of the earth's curve, light refraction remains somwhat of a mystery to me. The last three references above describe how this is calculated as a ratio. In other words, while the curving earth makes the horizon look slightly higher, light moving through air makes the horizon look lower. In ArcGIS, it is incorporated into the equation by reducing the effect's the earth's curve to only 6/7ths of the original curve. In other words, air usually lowers distant objects (from the observer's point of view) 1/7th of the distance that the earth's curve raises objects. Yoeli (1985:93) notes that this is applicable to "normal conditions."

1/7th = 0.13, hence this is the number you see in the Viewshed Tool's "Refractivity Coefficient" field.

I found a great photo essay explaining how this works, step by step, with equations showing exactly how much the earth's curvature and the refactions of air change an object's perceived height, using power plant stacks near Monterey Bay:

http://maps.unomaha.edu/Peterson/gis/Final_Projects/1998/Harmon/Final.html

For example, due to curvature, the stacks look 59 m lower. Due to refraction, they look 19 m higher. The problem is that the ratio of change due to air (19 m) to curvature (59 m) is not 1:7 (0.13), as the standard refraction coefficient suggests. It is 0.325.

Last spring I put together an Excel sheet that computes this ratio. Having the adjustable details (altitude, air pressure, wavelength, etc.) did show me that the ratio never really changes (given earthly conditions). What it did show me was that the ratio was always 0.325.

Should we be using 0.325 instead of 0.13? I'm still at a loss.

Fuzzy Viewsheds

As viewsheds get more refined, we need to model these things more accurately. A step in the right direction is Ogburn's (2006) update of Fisher's (1993) suggestion that viewsheds fade away over distance, or a fuzzy viewshed. In his example, viewshed fades away because 20/20 vision loses resolution over distance. The same equation, with different parameters, can be used to model any type of interference in viewshed that fade over distance. In any case, for many reasons, all viewsheds fade over distance. It seems that any responsible viewshed should incorporate this in some way. Ogburn (attched) did this in IDRISI, and I was able to get his equation to work in Arc, and will be soon posting those details.

Ogburn, Dennis E
2006 Assessing the Level of Visibility of Cultural Objects in Past Landscapes. Journal of Archaeological Science 33:405-413.

Any idea of how to calculate a non-standard refractivity coefficient would be great, not to mention how incorporate such variables into a fuzzy viewshed.

-Erik