block: Use bdrv_coroutine_enter to start I/O coroutines

BDRV_POLL_WHILE waits for the started I/O by releasing bs's ctx then polling
the main context, which relies on the yielded coroutine continuing on bs->ctx
before notifying qemu_aio_context with bdrv_wakeup().

Thus, using qemu_coroutine_enter to start I/O is wrong because if the coroutine
is entered from main loop, co->ctx will be qemu_aio_context, as a result of the
"release, poll, acquire" loop of BDRV_POLL_WHILE, race conditions happen when
both main thread and the iothread access the same BDS:

  main loop                                iothread
-----------------------------------------------------------------------
  blockdev_snapshot
    aio_context_acquire(bs->ctx)
                                           virtio_scsi_data_plane_handle_cmd
    bdrv_drained_begin(bs->ctx)
    bdrv_flush(bs)
      bdrv_co_flush(bs)                      aio_context_acquire(bs->ctx).enter
        ...
        qemu_coroutine_yield(co)
      BDRV_POLL_WHILE()
        aio_context_release(bs->ctx)
                                             aio_context_acquire(bs->ctx).return
                                               ...
                                                 aio_co_wake(co)
        aio_poll(qemu_aio_context)               ...
          co_schedule_bh_cb()                    ...
            qemu_coroutine_enter(co)             ...

              /* (A) bdrv_co_flush(bs)           /* (B) I/O on bs */
                      continues... */
                                             aio_context_release(bs->ctx)
        aio_context_acquire(bs->ctx)

Note that in above case, bdrv_drained_begin() doesn't do the "release,
poll, acquire" in BDRV_POLL_WHILE, because bs->in_flight == 0.

Fix this by using bdrv_coroutine_enter and enter coroutine in the right
context.

iotests 109 output is updated because the coroutine reenter flow during
mirror job complete is different (now through co_queue_wakeup, instead
of the unconditional qemu_coroutine_switch before), making the end job
len different.

Signed-off-by: Fam Zheng <famz@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
3 files changed
tree: 6a50c54236c0c103ed5b82c2fc362860630822ae
  1. audio/
  2. backends/
  3. block/
  4. bsd-user/
  5. chardev/
  6. contrib/
  7. crypto/
  8. default-configs/
  9. disas/
  10. docs/
  11. fpu/
  12. fsdev/
  13. gdb-xml/
  14. hw/
  15. include/
  16. io/
  17. libdecnumber/
  18. linux-headers/
  19. linux-user/
  20. migration/
  21. nbd/
  22. net/
  23. pc-bios/
  24. po/
  25. qapi/
  26. qga/
  27. qobject/
  28. qom/
  29. replay/
  30. roms/
  31. scripts/
  32. slirp/
  33. stubs/
  34. target/
  35. tcg/
  36. tests/
  37. trace/
  38. ui/
  39. util/
  40. .dir-locals.el
  41. .exrc
  42. .gitignore
  43. .gitmodules
  44. .mailmap
  45. .shippable.yml
  46. .travis.yml
  47. accel.c
  48. arch_init.c
  49. atomic_template.h
  50. balloon.c
  51. block.c
  52. blockdev-nbd.c
  53. blockdev.c
  54. blockjob.c
  55. bootdevice.c
  56. bt-host.c
  57. bt-vhci.c
  58. Changelog
  59. CODING_STYLE
  60. configure
  61. COPYING
  62. COPYING.LIB
  63. cpu-exec-common.c
  64. cpu-exec.c
  65. cpus-common.c
  66. cpus.c
  67. cputlb.c
  68. device-hotplug.c
  69. device_tree.c
  70. disas.c
  71. dma-helpers.c
  72. dump.c
  73. exec.c
  74. gdbstub.c
  75. HACKING
  76. hax-stub.c
  77. hmp-commands-info.hx
  78. hmp-commands.hx
  79. hmp.c
  80. hmp.h
  81. ioport.c
  82. iothread.c
  83. kvm-all.c
  84. kvm-stub.c
  85. LICENSE
  86. MAINTAINERS
  87. Makefile
  88. Makefile.objs
  89. Makefile.target
  90. memory.c
  91. memory_ldst.inc.c
  92. memory_mapping.c
  93. module-common.c
  94. monitor.c
  95. numa.c
  96. os-posix.c
  97. os-win32.c
  98. page_cache.c
  99. qapi-schema.json
  100. qdev-monitor.c
  101. qdict-test-data.txt
  102. qemu-bridge-helper.c
  103. qemu-doc.texi
  104. qemu-ga.texi
  105. qemu-img-cmds.hx
  106. qemu-img.c
  107. qemu-img.texi
  108. qemu-io-cmds.c
  109. qemu-io.c
  110. qemu-nbd.c
  111. qemu-nbd.texi
  112. qemu-option-trace.texi
  113. qemu-options-wrapper.h
  114. qemu-options.h
  115. qemu-options.hx
  116. qemu-seccomp.c
  117. qemu-tech.texi
  118. qemu.nsi
  119. qemu.sasl
  120. qmp.c
  121. qtest.c
  122. README
  123. replication.c
  124. replication.h
  125. rules.mak
  126. softmmu_template.h
  127. spice-qemu-char.c
  128. tcg-runtime.c
  129. tci.c
  130. thunk.c
  131. tpm.c
  132. trace-events
  133. translate-all.c
  134. translate-all.h
  135. translate-common.c
  136. user-exec-stub.c
  137. user-exec.c
  138. VERSION
  139. version.rc
  140. vl.c
  141. xen-common-stub.c
  142. xen-common.c
  143. xen-hvm-stub.c
  144. xen-hvm.c
  145. xen-mapcache.c