Clogged hair follicles are more likely to occur in areas that are frequently subjected to friction. Over time, they can cause pain, infection, or scarring. Therefore, when painful swelling due to clogged hair follicles recurs, you should see a doctor for advice on appropriate treatments to help your skin look healthier and firmer.