Cut the apricots each into four pieces and place in a bowl of warm water with the brandy (if you are using it).
Pour the milk into another mixing bowl and add the torn bread. Push the bread down into the milk and mix it a bit with a wooden spoon to get it all soaking. Place this in the refrigerator to soak for 1 hour.
In a mixing bowl, whisk the 4 eggs, then add the sugar and vanilla extract, whisking until combined. Pour this on top of the soaked bread, and stir to combine. Drain the water from the apricots and stir them in.
Pour the mixture into a well greased 9 x 13 inch baking pan. Dot with small pieces of butter.
Bake at 325° F for 1 hour or until golden brown and set. Serve warm or cold.
Makes 6 - 8 servings.