Start by placing 12 Oreo cookies on a baking sheet, cream side up.
Scoop about 1/4 cup of softened vanilla ice cream onto each cookie.
If desired, sprinkle chocolate chips and mini marshmallows on top of the ice cream.
Place the remaining 12 Oreo cookies on top of the ice cream, cream side down, to create sandwiches.
Gently press down on each sandwich to ensure the ice cream spreads to the edges.
Place the baking sheet in the freezer for at least 1 hour to allow the ice cream to firm up.
Once frozen, remove the sandwiches from the freezer and cut them in half for easier serving, if desired.