Human infertility is a significant global health concern that impacts the lives of numerous couples attempting to conceive. A substantial proportion of infertility cases are attributed to male-related factors. A pivotal component in the diagnosis of male infertility is semen analysis, which heavily depends on the evaluation of sperm head morphology, particularly the shape and size of the sperm head. In many instances, this evaluation is conducted manually, with its accuracy relying on the expertise and experience of the physician. This manual process is not only time-intensive and laborious but also subject to considerable variations depending on different laboratories or physicians within the same laboratory. Such variation poses challenges in ensuring consistent and reliable diagnostic outcomes. This paper presents a hybrid deep learning-based framework designed to enable automated and accurate classification of human sperm heads based on sperm images. Experimental results obtained from two datasets, HuSHeM and SCIAN, demonstrate the proposed framework's superior effectiveness in analyzing and classifying sperm head morphology compared to existing methodologies.