This happens not only on my personal projects, but even attempting to use the bake test scene Marmoset provides. The program crashes to desktop almost immediately. I increased my TDR Delay and I reduced my Baker GPU Priority to Responsive but I'm still having crashes occur. 
Processor: Intel i7-8700 
RAM: 32.0gb
Display: Radeon RX 5700
 
            
Replies
Work Processor: Intel I9-10920X
RAM: 32.0gb
Display: Geforce RTX 2070 Super
I have a raging hunch that Radeon's most recent driver update may be the underlying cause, as I've baked on my home system before without issue.