Image downsampling changes the image range. For example, if you have an image of size 4×44\times 44×4.
I0=(16151413121110987654321) I_0= \left( \begin{aligned} 16 && 15 && 14 && 13\\ 12 && 11 && 10 && 9\\ 8 && 7 && 6 && 5\\ 4 && 3 && 2 && 1 \end{aligned} \right) I0=16128415117314106213951
If we downsampling the image into a size of 2×22\times22×2 by taking I0[0:2:end,0:2:end]I_0[0:2:end, 0:2:end]I0[0:2:end,0:2:end], then the result becomes
I1=(161486) I_1= \left( \begin{aligned} 16 && 14\\ 8 && 6 \end{aligned} \right) I1=(168146)
Now the range of I1I_1I1 is [6,16][6, 16][6,16], while the range of I0I_0I0 is [1,16][1, 16][1,16]. Correspondingly, if you normalize I1I_1I1 based on one equation
I1,norm=I1−min(I1)max(I1)−min(I1) I_{1,norm}= \frac{I_1-\min(I_1)}{\max(I_1)-\min(I_1)} I1,norm=max(I1)−min(I1)I1−min(I1)
The scale of I1I_1I1 would differ from direct normalization upon I0I_0I0. And once you denormalize the real value out of the wrong result, severe mistakes would occur.