Highest precision of numerical differentiation

One of the very important question on numerical differentiation, is precision of the results. What is highest possible precision of numerical differentiation, what factors can limit this precision and how can we improve it.

RMSD for measuring precision of numerical differentiation

The best way to check the precision of calculation is to compare analytically calculated derivative with numerically calculated. Best measure is RMSD, or Root Mean Square Difference. The RMSD is calculating by equation rmsd = sqrt((1/n)*Σ₁^Nd²), where d – is the distance between equivalents dots and N – number of dots. In one dimensional case, the RMSD between two numpy array can be calculated by this simple code:

def calc_rmsd(a, b):
    return (((a-b)**2).mean())**0.5

Increasing precision for numerical differentiation

Difference schemes of numerical differentiation can give different precision for the same differentiation step size. Also reducing of this differentiation step will give better accuracy of calculations. But reducing of differentiation step, it is always necessary to remember about very important limiting factor.

Machine accuracy

It is impossible to reduce differentiation step below machine accuracy. For standard double (or float64) data you have only 52 bit to store your fraction. And these 52 bits should be able to store value of x+dx. And also during any mathematical operations, the errors tent to be increase.

In this video, you can see the achievement of the highest possible accuracy for different schemes of numerical differentiation with python.

Script to estimate accuracy

To estimate highest possible accuracy, it is necessary to perform numerical differentiation with different steps and calculate RMSD between numerical and analytical solution.

The main conclusion, is that for CD4 (central difference scheme of the fourth order) the minimal step for differentiation cannot be smaller than 1/10⁴ of them maximal value of variable for the function to be dedifferentiated. And for higher, more accurate schemes this value can be even bigger!!!

So in fact, I would like to advise to keep you differentiation step higher than 5/10⁴ of the maximal value of variable.

Published: 2022-09-04 23:34:25