Preheat oven to 350°F (175°C) and line or grease a 12-cup muffin tin.
Whisk together flour, baking powder, baking soda, salt, cinnamon, nutmeg, cloves, and ginger.
In another bowl, mix pumpkin purée, oil, brown sugar, milk, eggs, and vanilla until smooth.
Fold dry ingredients into wet just until combined.
Mix brown sugar, cinnamon, and melted butter for the swirl.
Fill muffin cups halfway with batter, add a teaspoon of swirl, cover with more batter to ¾ full, and swirl gently with a toothpick.
Bake 16–18 minutes until a toothpick comes out clean.
Cool 5 minutes in the pan, then transfer to a rack. Whisk powdered sugar, milk, and vanilla for glaze and drizzle over slightly warm muffins.