Preheat oven to 350 degrees.
Grease and flour a 9x5 inch loaf pan
Mix the brown sugar and cinnamon together in a bowl and set aside.
In bowl of stand mixer, or large bowl using hand mixer, beat the butter for one minute until smooth and creamy.
Add the sugar and beat for one to two minutes until fluffy.
Add the eggs and vanilla, and continue to beat until combined.
In a separate medium bowl combine the flour, baking powder, cinnamon, and nutmeg with a whisk until combined.
Add the dry ingredients to wet ingredients, along with the milk, and mix to combine.
Pour half of the batter into the prepared pan, cover with half of the chopped apples, slightly pressing them into the batter, and then sprinkle with half of the cinnamon brown sugar mixture.
Pour the remaining batter over the apple/cinnamon sugar layer, top with the remaining apples and remaining cinnamon sugar mixture.
Pat the topping down slightly into the batter.
Bake for 45-50 minutes, until toothpick inserted in center comes out clean.
Cool in pan for 10 minutes before transferring bread to a cooling rack.
In a small bowl using a hand mixer or whisk, comine the powdered sugar, vanilla, cinnamon, and milk until combined and smooth.
Add more milk as necessary to reach desired consistency.
Drizzle glaze over bread as desired.