So I generated supports with the Internal Support option off and I noticed two tiny islands that needed internal support. So I entered the Edit Supports mode, added support points to the areas needed, but it did not generate any supports there.
So instead I had to generate supports with internal supports on and literally delete hundreds of points instead of two simple clicks. Seems like a bug?