In Block Editor, the image captions don't look like captions. They look like regular paragraph text. Please style the captions so that they look like the captions in the WYSIWYG Editor:
  • Italic
  • Slightly smaller font size than the p tag
Attaching a screenshot to show the difference between the captions in the 2 editors.