Ok, thank yor for advice. Yeah,I understand it. 8 bit height map is limited to 255 step info, what is not a lot if I want more details, 16 bit would give me 65535 info, what is a bigger span for height map. I have to find high resolution map source with at least 16 bit data.
I thought if I converted 8 bit map into 16 in Photoshop, Photoshop would automatically fill in the gaps between pixel values, some average value between pixels and would give more data to my dem map, made surface more smooth. But I can achive same thing by adding turbosmooth or decay, blur in displacement modifier.