The Epic Games Launcher shows a thumbnail for each project, looking in two places: first in the root folder of the game for [ProjectName].png (next to the .uproject) and then Saved\AutoScreenshot.png, which is created by the editor automatically. You can create your own thumbnail (192×192 .png) and set it using the “Project Thumbnail” option in the Description category of Project Settings.
You can also customize the splash screen shown when the editor or game is starting up (separately for editor vs game, windows vs mac) in the Windows / Mac categories (under Platform) of Project Settings. The splash screens need to be .bmp (not .png), but they can be whatever size you want within reason (the startup text and project name are overlaid on top of it, so you’ll probably want to leave space for that).
You can pick application icons and mobile startup screens in the various platform settings options pages as well (the rules for these are varied and some platforms require a wide range of sizes, but it’s all listed in the corresponding settings pages and platform vendor web sites). Due to the variety of required splash screens for a universal iOS app, the size can start to add up noticeably, so it can be worthwhile to approach them differently, using large fill areas that will compress well, run pngcrush on them, etc….