Heat oven to 375F and grease 9x13 casserole dish.
Add diced brioche to a large bowl, add chopped apples and pecans.
In a separate bowl, whisk together eggs, milk, 1/2 cup brown sugar, cinnamon, nutmeg, vanilla, and salt.
Pour egg mixture over bread mixture and toss until evenly coated, but do not over mix, as you don't want it to become soggy.
Pour mixture into greased casserole dish.
In a small bowl, melt butter in microwave. Whisk in remainder of brown sugar once melted, and then pour this mixture over top of french toast mixture.
Bake for 70-80 minutes or until brown and crispy on top.
Let cool 10 minutes before serving.