Construction of the Jami-Ul-Alfar Mosque commenced in 1908 and the building was completed in 1909. The mosque was commissioned by the local Indian Muslim community, based in Pettah, to fulfill their required five-times-daily prayer and Jummah on Fridays. The mosque's designer and builder was Habibu Labbe Saibu Labbe (an unqualified architect), and was based on details/images of Indo-Saracenic structures provided by South Indian traders, who commissioned him. It is a hybrid style of architecture, that draws elements from native Indo-Islamic and Indian architecture, and combines it with the Gothic revival and Neo-classical styles. Originally it had the capacity for 1,500 worshippers although at the time only around 500 were attending prayers.

It is a distinctive red and white candy-striped two-storey building, with a clock tower, and is reminiscent of the Jamek Mosque in Kuala Lumpur, Malaysia (constructed in 1910). Before other landmarks were built, some claim that the Jami Ul-Alfar Mosque was recognized as the landmark of Colombo by sailors approaching the port.

In 1975 the mosque, with the assistance of the Haji Omar Trust, purchased a number of the adjoining properties and commenced building an expansion to the mosque to increase its capacity to 10,000.