The little big difference between Vips::Image#thumbnail_image and Vips::Image::thumbnail

[Nakilon]

The issue

I am comparing web page screenshots for testing purposes. The workflow is like this:

1. obtain the screenshot blob from chromium and load it via Image.new_from_buffer
2. assert the similarity with the reference.jpg asset and if it's significant, fail the test
3. if in fact there was no error, update the asset, manually rewriting it with the saved blob

The issue is that even after I renew the asset file it fails again (with the perceptual difference of 11), BUT if I save the current blob to another file and then load it back, the perceptual difference is zero!

Why? Because of some unexpected (for me and for practical reason, because it looks stupid to fix it via save/load) difference that exists between Vips::Image#thumbnail_image and Vips::Image::thumbnail here:

      image = if input.is_a? Vips::Image
        input.thumbnail_image(size, height: size, size: :force)
      else
        Vips::Image.thumbnail(input, size, height: size, size: :force)
      end

The load-from-buffer feature was wanted by users two, times, and you would not notice the issue if all your fingerprinting was either from Image or from buffer, but in my workflow I need to compare practically the same blob loaded in both ways, and that's where the difference occures.

How to observe it?

vk_video.jpg.zip

blob = File.binread "vk_video.jpg"

puts Vips::Image.new_from_buffer(blob, "").thumbnail_image(8, height: 8, size: :force).
  colourspace("b-w")[0].to_a.map{ |_| _.map{ |_| "%3d" % _ }.join }

 47 42 47 50 47 45 46 47
  4  5  3  3  4  4  4  4
  0  0  0  0  0  0  0  0
  1  1  1  2  5  1  1  1
  0  0  0  0  0  0  0  0
103102 97101 99 99102111
248243243247251251250254
246238239254253253253252

puts Vips::Image.thumbnail("vk_video.jpg", 8, height: 8, size: :force).
  colourspace("b-w")[0].to_a.map{ |_| _.map{ |_| "%3d" % _ }.join }

 45 42 45 47 46 44 45 46
  4  4  4  3  3  4  4  4
  0  0  0  0  1  0  0  0
  4  4  4  5  8  4  4  4
  0  0  0  0  0  0  0  0
104102 99103 98 99101111
250247248250255255254254
245239241253253253253252

Or use this debug script:

require "dhash-vips"

ha = DHashVips::IDHash.fingerprint Vips::Image.new_from_buffer File.binread("vk_video.jpg"), ""
hb = DHashVips::IDHash.fingerprint "vk_video.jpg"

puts "distance: #{d1 = DHashVips::IDHash.distance ha, hb}"
size = 2 ** 3
shift = 2 * size * size
ai = ha >> shift
ad = ha - (ai << shift)
bi = hb >> shift
bd = hb - (bi << shift)

_127 = shift - 1
_63 = size * size - 1

d2 = 0
a, b = [[ad, ai], [bd, bi]].map do |xd, xi|
  puts ""
  hor = Array.new(size){Array.new(size){" "}}
  ver = Array.new(size){Array.new(size){" "}}
  _127.downto(0).each_with_index do |i, ii|
    if i > _63
      y, x = (_127 - i).divmod size
    else
      x, y = (_63 - i).divmod size
    end
    if xi[i] > 0
      target, c = if i > _63
        [ver, %w{ v ^ }[xd[i]]]
      else
        [hor, %w{ > < }[xd[i]]]
      end
      target[y][x] = c
    end
    if ai[i] + bi[i] > 0 && ad[i] != bd[i]
      d2 += 1
      target = if i > _63
        ver
      else
        hor
      end
      target[y][x] = "\e[7m#{target[y][x]}\e[27m"
    end
  end
  hor.zip(ver).each{ |_| puts _.join " " }
end

Expected behavior

The difference to be ≤5.

Screenshots

Environment:

• vips-8.11.3-Wed Aug 11 09:29:27 UTC 2021
• ruby-vips 2.2.1

---

I could rework the fingerprinting algorithm specifically for such case (the case of non-photorealistic images with huge blank lines) by calculating the "significance median" not two times (for vertical and horizontal half of fingerprinting) but once for both added, and maybe I will do it but not soon. I just think you might want to double-check if there is something unexpected happening in libvips.

[jcupitt]

Hi @Nakilon,

Yes, thumbnail and thumbnail_image will give different results since they work in different ways. thumbnail_image is only there for emergencies and should really not be used -- it will often be much, much slower.

I would use thumbnail_buffer instead -- it's as fast as thumbnail_image and should give exactly the same pixels.

The main docs warn against thumbnail_image:

https://www.libvips.org/API/current/libvips-resample.html#vips-thumbnail-image

Though maybe the ruby docs should have this warning too :(

The dev checklist in the next version of the docs has some more info:

https://github.com/libvips/libvips/blob/master/doc/Developer-checklist.md

[taylorthurlow]

I would use thumbnail_buffer instead -- it's as fast as thumbnail_image and should give exactly the same pixels.

Just so I understand this correctly - thumbnail_image is not ideal, and you recommend thumbnail_buffer instead? In what way is thumbnail_buffer preferable if it's not actually any faster and gives you the same pixels? I do understand why you would want to use a thumbnail method that can leverage shrink-on-load, but if the use case means that's not an option, would it still be preferable to use thumbnail_buffer over thumbnail_image?

Though maybe the ruby docs should have this warning too :(

FWIW I did run into this footgun in a few cases and solved a lot of problems by switching to thumbnail_source.