vmware_vga: fix out of bounds and invalid rects updating

This is a follow up for several attempts to fix this issue.

Previous incarnations:

1. http://thread.gmane.org/gmane.linux.ubuntu.bugs.general/3156089
https://bugs.launchpad.net/bugs/918791
"qemu-kvm dies when using vmvga driver and unity in the guest" bug.
Fix by Serge Hallyn:
 https://launchpadlibrarian.net/94916786/qemu-vmware.debdiff
This fix is incomplete, since it does not check width and height
for being negative.  Serge weren't sure if that's the right place
to fix it, maybe the fix should be up the stack somewhere.

2. http://thread.gmane.org/gmane.comp.emulators.qemu/166064
by Marek Vasut: "vmware_vga: Redraw only visible area"

This one adds the (incomplete) check to vmsvga_update_rect_delayed(),
the routine just queues the rect updating but does no interesting
stuff.  It is also incomplete in the same way as patch by Serge,
but also does not touch width&height at all after adjusting x&y,
which is wrong.

As far as I can see, when processing guest requests, the device
places them into a queue (vmsvga_update_rect_delayed()) and
processes this queue in different place/time, namely, in
vmsvga_update_rect().  Sometimes, vmsvga_update_rect() is
called directly, without placing the request to the gueue.
This is the place this patch changes, which is the last
(deepest) in the stack.  I'm not sure if this is the right
place still, since it is possible we have some queue optimization
(or may have in the future) which will be upset by negative/wrong
values here, so maybe we should check for validity of input
right when receiving request from the guest (and maybe even
use unsigned types there).  But I don't know the protocol
and implementation enough to have a definitive answer.

But since vmsvga_update_rect() has other sanity checks already,
I'm adding the missing ones there as well.

Cc'ing BALATON Zoltan and Andrzej Zaborowski who shows in `git blame'
output and may know something in this area.

If this patch is accepted, it should be applied to all active
stable branches (at least since 1.1, maybe even before), with
minor context change (ds_get_*(s->vga.ds) => s->*).  I'm not
Cc'ing -stable yet, will do it explicitly once the patch is
accepted.

BTW, these checks use fprintf(stderr) -- it should be converted
to something more appropriate, since stderr will most likely
disappear somewhere.

Cc: Marek Vasut <marex@denx.de>
CC: Serge Hallyn <serge.hallyn@ubuntu.com>
Cc: BALATON Zoltan <balaton@eik.bme.hu>
Cc: Andrzej Zaborowski <balrogg@gmail.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Marek Vasut <marex@denx.de>
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
1 file changed
tree: 4d1e76afbf538b1c5d3daa132d2ae74bf742c364
  1. audio/
  2. backends/
  3. block/
  4. bsd-user/
  5. default-configs/
  6. disas/
  7. docs/
  8. fpu/
  9. fsdev/
  10. gdb-xml/
  11. hw/
  12. include/
  13. ldscripts/
  14. libcacard/
  15. linux-headers/
  16. linux-user/
  17. net/
  18. pc-bios/
  19. qapi/
  20. qga/
  21. QMP/
  22. qobject/
  23. qom/
  24. roms/
  25. scripts/
  26. slirp/
  27. stubs/
  28. sysconfigs/
  29. target-alpha/
  30. target-arm/
  31. target-cris/
  32. target-i386/
  33. target-lm32/
  34. target-m68k/
  35. target-microblaze/
  36. target-mips/
  37. target-openrisc/
  38. target-ppc/
  39. target-s390x/
  40. target-sh4/
  41. target-sparc/
  42. target-unicore32/
  43. target-xtensa/
  44. tcg/
  45. tests/
  46. trace/
  47. ui/
  48. util/
  49. .exrc
  50. .gitignore
  51. .gitmodules
  52. .mailmap
  53. aio-posix.c
  54. aio-win32.c
  55. arch_init.c
  56. async.c
  57. balloon.c
  58. block-migration.c
  59. block.c
  60. blockdev-nbd.c
  61. blockdev.c
  62. blockjob.c
  63. bt-host.c
  64. bt-vhci.c
  65. Changelog
  66. cmd.c
  67. cmd.h
  68. CODING_STYLE
  69. configure
  70. COPYING
  71. COPYING.LIB
  72. coroutine-gthread.c
  73. coroutine-sigaltstack.c
  74. coroutine-ucontext.c
  75. coroutine-win32.c
  76. cpu-exec.c
  77. cpus.c
  78. cputlb.c
  79. device_tree.c
  80. disas.c
  81. dma-helpers.c
  82. dump-stub.c
  83. dump.c
  84. exec.c
  85. gdbstub.c
  86. HACKING
  87. hmp-commands.hx
  88. hmp.c
  89. hmp.h
  90. iohandler.c
  91. ioport.c
  92. kvm-all.c
  93. kvm-stub.c
  94. LICENSE
  95. main-loop.c
  96. MAINTAINERS
  97. Makefile
  98. Makefile.objs
  99. Makefile.target
  100. memory.c
  101. memory_mapping-stub.c
  102. memory_mapping.c
  103. migration-exec.c
  104. migration-fd.c
  105. migration-tcp.c
  106. migration-unix.c
  107. migration.c
  108. monitor.c
  109. nbd.c
  110. os-posix.c
  111. os-win32.c
  112. page_cache.c
  113. qapi-schema-test.json
  114. qapi-schema.json
  115. qdict-test-data.txt
  116. qemu-bridge-helper.c
  117. qemu-char.c
  118. qemu-coroutine-io.c
  119. qemu-coroutine-lock.c
  120. qemu-coroutine-sleep.c
  121. qemu-coroutine.c
  122. qemu-doc.texi
  123. qemu-img-cmds.hx
  124. qemu-img.c
  125. qemu-img.texi
  126. qemu-io.c
  127. qemu-log.c
  128. qemu-nbd.c
  129. qemu-nbd.texi
  130. qemu-options-wrapper.h
  131. qemu-options.h
  132. qemu-options.hx
  133. qemu-seccomp.c
  134. qemu-tech.texi
  135. qemu-timer.c
  136. qemu.sasl
  137. qmp-commands.hx
  138. qmp.c
  139. qtest.c
  140. readline.c
  141. README
  142. rules.mak
  143. savevm.c
  144. spice-qemu-char.c
  145. tcg-runtime.c
  146. tci.c
  147. thread-pool.c
  148. thunk.c
  149. TODO
  150. trace-events
  151. translate-all.c
  152. translate-all.h
  153. user-exec.c
  154. VERSION
  155. version.rc
  156. vl.c
  157. xen-all.c
  158. xen-mapcache.c
  159. xen-stub.c