Mix together flour, cocoa, baking soda, and salt. Cream together butter and sugar until light and creamy. Add egg, milk, and vanilla. Add the dry ingredients. Mix until combined. Line baking sheets with parchment paper. Using an ice cream scoop, drop batter about 2 inches apart.
Bake in a 350 degree oven for 10 to 12 minutes. Take the cookies out and press the marshmallow half into the cookie. Put the cookies back into the oven for another 2 minutes. Transfer to a cooling rack.
Make the frosting by whisking together sugar, butter, cocoa, milk, and vanilla. Top with frosting and cover the marshmallow.