Spread 3 tablespoons butter on 1 side of bread slices. Toast bread, buttered side down, in a large skillet or electric griddle over medium-low heat until golden brown, 2-3 minutes; remove. Strain 1 teaspoon of chili oil from the chili crisp. Add to a small bowl, along with mayonnaise, Parmesan and remaining 3 tablespoons butter; stir until well combined.
To assemble sandwiches, top toasted side of 4 bread slices with a slice of cheddar cheese. Drizzle each with remaining chili crisp. Top with slice of mozzarella cheese. Top with remaining bread slices, toasted side facing inward. Spread mayonnaise mixture on the outsides of each sandwich.
Place sandwiches in same skillet, and cook until bread is golden brown and cheese is melted, 3-4 minutes on each side. Serve immediately.