Some good news: Both cards are now working (albeit only with a 18MHz crystal and firmware). One had a cold solderjoint and dead CPLD (reset got stuck at 3.7V), no idea what's up with the other, it simply started working after a few attempts.
Output is still kinda distorted, but loading and playback works reliable - I'll check what's up with the output later on. Tested it for 30 minutes with Zynaps (which borrows 1 channel for sound effects).