In a mixing bowl, whisk together pudding and milk until thickened. Place in fridge and allow to set.
In another bowl, beat together the softened cream cheese, butter, powdered sugar, and vanilla extract until smooth and creamy.
Gently fold the prepared pudding mixture into the cream cheese mixture until well combined.
Add whipped topping and mix carefully until mixture is smooth and fully incorporated.
Place the chocolate sandwich cookies in a large zip-top bag and crush them using a rolling pin or the bottom of a heavy glass until they resemble dirt.
In individual serving cups or a large serving bowl, layer the pudding mixture and crushed cookies alternately, starting and ending with the crushed cookies. This creates the "dirt" layers.
If desired, place gummy worms on top of the pudding.
Refrigerate at least an hour before serving.