There actually is a little overlap in Units 1&2 and 3&4, only brief mentions, though.
In Unit 1, we learned a Unit 4 topic which was the 4P Factor Model, which is basically like an alternate/extended form of the biopsychosocial framework. But there is plenty of variety in Units 3&4, and is more than just a mere extension from 1&2. I wouldn't say it's necessarily harder, but there is plenty to learn if you really do like to study how the mind works.
And of course, if it's a subject you don't want to do, then, please drop it. As Jordan Peterson said (and I'm paraphrasing), there's no point in suffering through anything if there is no useful desired result at the end. The semester is coming to a close soon, so try to slog through the exams and if you so wish, change your subjects.