Beat cream cheese, pumpkin pie spice and 1/4 cup of the sugar in large bowl with electric mixer on medium speed until smooth and creamy. Add coconut milk; beat until well blended.
Beat cream and remaining 1/4 cup sugar in another large bowl with electric mixer on medium speed until stiff peaks form. Add 1/2 of the whipped cream to cream cheese mixture; stir until well blended. Gently stir in remaining whipped cream.
Divide brownie pieces evenly among each of 12 (2-ounce) shot glasses. Layer each glass with fudge sauce and mousse. Garnish with toasted sliced almonds, if desired.