Cut brioche in 2-inch blocks. Place the brioche in a casserole, baking dish or deep container. Sprinkle grapefruit, lime, and lemon peel and pour cream and coconut milk into the dish. Let bread marinate overnight in the refrigerator, flipping the bread once halfway through.
Heat 1 tablespoon butter in a nonstick saute pan over medium heat. Remove bread from marinade, discarding liquids and fruit peels. Sprinkle half the sugar on the brioche and place the bread on the pan. Brown the bread on all sides until golden. Remove from heat.
Sprinkle remaining sugar on the bread. Use a blow torch to carefully caramelize the brioche. Alternatively, preheat broiler to medium. Place bread in the broiler and cook until sugars have melted and caramelized, about 1-2 minutes. Be sure to keep an eye on the bread to make sure it doesn't burn. Serve immediately.