Preheat the oven to 350°F and grease a Twinkie or mini loaf pan. In one bowl, whisk together flour, baking powder, and salt.
In another, cream butter with sugar until fluffy, then beat in eggs and vanilla. Add the dry mixture and milk alternately, mixing just until combined. Fill each mold about ¾ full.
Bake for 18–22 minutes, until a toothpick comes out clean. Cool briefly in the pan, then move to a wire rack to cool completely.
In a saucepan, cook diced peaches with sugar, cornstarch, and lemon juice over medium heat, stirring until softened and thickened. Let cool to room temperature.
Hollow out the center of each cooled cake with a knife or skewer, then spoon or pipe peach filling inside.
Whisk powdered sugar, cream or milk, and vanilla until smooth. Drizzle over the cakes and let set before serving.