Assumed Position: adjustment necessary when using calculator?

[Stearmandriver]

Hello all! 

Great forum here. I've been browsing and learning for a week; as esoteric as it might be these days, it's nice to have a dedicated place for this subject, for people like me that are just learning. 

I have a question I wanted to run by you.  I've done a few manual sight reductions using the tables and a Davis Mark 3 with a bowl of water for an artificial horizon, and been getting LOPs with accuracy within a few miles which I feel is acceptable for a beginner with such a hokey setup ;).  But I've also been using an online sight reduction calculator just to make the practice process faster (yes it's cheating, but who doesn't just want to play with a new toy for a bit?)  I've been using this calculator: 
https://www.celnav.de/sightred.htm

Along with the USNO almanac data.  Seems to work well; again,  LOPs within a few miles.  But I'm curious:  using the tables, it seems (at least with the tables and forms I've been using) necessary to adjust your initial Assumed Position longitude during the conversion to LHA.  This "adjusted" AP longitude is then used in the LOP plotting process.  

So: is it necessary to also adjust the longitude of the Assumed Position when using that online calculator? I'm guessing not, because, well, plotting the LOP using the calculated intercept from the original AP seems to work well, and the calculator doesn't supply a modified AP longitude.  I'm *guessing* that since this is a calculator, it's solving the appropriate formulae using the actual AP, without any modification being driven by the tables. 

Sanity check? Am I right to plot the LOP directly from my original assumed position when using this online calculator? 

Thanks!

[BigBill]

I don't know the answer to your question but maybe you could take several sightings and figure them both ways. if your results are close you might have your answer.

[Stearmandriver]

I don't know the answer to your question but maybe you could take several sightings and figure them both ways. if your results are close you might have your answer.

Yup, that occurred to me after I wrote this.  I've tried plotting 2 sightings reduced by tables and the resulting LOPs are less than .2nm away from the ones the calculator returned, so I'd say the calculator is running its calculations using the actual AP.  It seems the modification in the tables (249) is to absorb the error from rounding LHA to a whole degree value? So I suppose, if you're doing the actual trig (the way a calculator does), there's no need to round. I suppose, also, this makes the calculators results a little more accurate than the tables... but if we're talking two tenths of a mile, I'm not gonna be too concerned.

Thanks!

[Rumata]

Hello all! 

Great forum here. I've been browsing and learning for a week; as esoteric as it might be these days, it's nice to have a dedicated place for this subject, for people like me that are just learning. 

I have a question I wanted to run by you.  I've done a few manual sight reductions using the tables and a Davis Mark 3 with a bowl of water for an artificial horizon, and been getting LOPs with accuracy within a few miles which I feel is acceptable for a beginner with such a hokey setup ;).  But I've also been using an online sight reduction calculator just to make the practice process faster (yes it's cheating, but who doesn't just want to play with a new toy for a bit?)  I've been using this calculator: 
https://www.celnav.de/sightred.htm

Along with the USNO almanac data.  Seems to work well; again,  LOPs within a few miles.  But I'm curious:  using the tables, it seems (at least with the tables and forms I've been using) necessary to adjust your initial Assumed Position longitude during the conversion to LHA.  This "adjusted" AP longitude is then used in the LOP plotting process.  

So: is it necessary to also adjust the longitude of the Assumed Position when using that online calculator? I'm guessing not, because, well, plotting the LOP using the calculated intercept from the original AP seems to work well, and the calculator doesn't supply a modified AP longitude.  I'm *guessing* that since this is a calculator, it's solving the appropriate formulae using the actual AP, without any modification being driven by the tables. 

Sanity check? Am I right to plot the LOP directly from my original assumed position when using this online calculator? 

Thanks!

Greetings,

A few times I tried to use exact DR longitude, meaning with degrees and minutes , not assumed position longitude, just to see what the difference will it make. It was anti-sanity check, to use interpolation between different pages of Pub.229 to get not a significant difference in the result. Both Tables and calculators are using sin/cos spherical trigonometry formulas. Therefore, I would not expect big difference. Anyway, I think eons-old method of rounding DR longitude to get whole number LHA/t is working. My HP calculator a few times showed me that 27+13=54.3. It is good that I passed my PE test years ago. So I was still able to catch a slight imperfection in the result. So much for calculators.

[jeremyparker]

Hello all! 

Great forum here. I've been browsing and learning for a week; as esoteric as it might be these days, it's nice to have a dedicated place for this subject, for people like me that are just learning. 

I have a question I wanted to run by you.  I've done a few manual sight reductions using the tables and a Davis Mark 3 with a bowl of water for an artificial horizon, and been getting LOPs with accuracy within a few miles which I feel is acceptable for a beginner with such a hokey setup ;).  But I've also been using an online sight reduction calculator just to make the practice process faster (yes it's cheating, but who doesn't just want to play with a new toy for a bit?)  I've been using this calculator: 
https://www.celnav.de/sightred.htm

Along with the USNO almanac data.  Seems to work well; again,  LOPs within a few miles.  But I'm curious:  using the tables, it seems (at least with the tables and forms I've been using) necessary to adjust your initial Assumed Position longitude during the conversion to LHA.  This "adjusted" AP longitude is then used in the LOP plotting process.  

So: is it necessary to also adjust the longitude of the Assumed Position when using that online calculator? I'm guessing not, because, well, plotting the LOP using the calculated intercept from the original AP seems to work well, and the calculator doesn't supply a modified AP longitude.  I'm *guessing* that since this is a calculator, it's solving the appropriate formulae using the actual AP, without any modification being driven by the tables. 

Sanity check? Am I right to plot the LOP directly from my original assumed position when using this online calculator? 

Thanks!

When you use tables to calculate the altitude at your Assumed Position it is necessary to choose a position that results in an LHA and Latitude as a whole number of degrees. This is because tables need to be concise, and allowing for incremental values between degrees of Lat or Long would result in the tables being enormous and impracticable.
Because of this necessity to choose an AP that is not your actual Estimated Position, you may end up with a large intercept.

However, as you correctly surmise, the online calculator is calculating the altitude using a formula, so is able to use your Estimated Position - that is to say, the position you actually suppose yourself to be in, rather than a position that you must assume for the purpose of using the tables.

If you use the online calculator to check your reduction of a sight using Sight Reduction tables, and you enter the same data into each, then you should obtain a similar intercept and azimuth, and you would work your intercept from the same ASSUMED Position.
On the other hand, if you wish to compare the precision of the two methods, you could reduce the sight using tables and plot an intercept from the AP, and then you could enter your EP into the calculator and plot the resultant intercept from that EP. Whilst the values of the two intercepts will differ, the result should be essentially the same, that is to say, the two LOPs should be nearly identical. I say nearly identical because there may be a small difference in azimuth, this being because the body may lie in slightly different directions from the two positions. Other small differences will result from the greater precision of the corrections used by the online calculator (corrections for temperature and pressure, as well as precise corrections for semi-diameter and HP whereas the almanac uses average values for these) and from the rounding of values in the tables.

Bottom line, reducing a sight using a formula is more exact. But, for beginners and/or occasional users, using tables and a short form may result in fewer errors and any such errors are more easily traceable. One may generally say that, for a sight obtained at near sea level from the deck of a small vessel any reduction in accuracy occasioned by use of tables is still within acceptable tolerances.