Arrange graham crackers in a single layer on the bottom of a 9×13 inch baking dish, breaking as needed to fit.
In a large bowl, whisk together the two boxes of instant vanilla pudding mix with 3 cups of milk. Continue whisking until thick and set, about 2 minutes.
Gently fold the thawed whipped topping into the thickened pudding mixture using a spatula until no white streaks remain.
Spread half of the pudding mixture over the graham cracker layer in the dish.
In another bowl, combine the powdered sugar, softened butter, remaining 1/3 cup milk, unsweetened cocoa powder, and vanilla extract. Beat until smooth and thick.
Spread the chocolate frosting evenly over the first layer of pudding.
Top with the remaining pudding mixture and press gently to create smooth layers.
Refrigerate overnight, or for at least 8 hours, to allow the graham crackers to soften and the flavors to meld.
Serve chilled and enjoy!