For crab cakes: Place all the crab cake ingredients, except the crab meat, into a large mixing bowl and mix together.
Season with salt and pepper and gently fold in the crab meat until just combined.
Separately place flour, egg and panko into shallow baking dishes.
Form 4 equally sized crab cakes with the mixture and dredge in the flour, followed by the egg and finally the panko breadcrumbs, shaking off any excess after each dip.
Lightly season each coated crab cake with salt and pepper and set aside.
Pour two tablespoons oil into a large skillet and place over medium heat.
Place each crab cake into the hot skillet, about 1 inch apart. Pan fry on each side for 4 to 5 minutes or until golden brown and cooked through.
Transfer crab cakes onto a plate lined with a paper towel. Lightly season with salt and pepper.
For garlic aioli: Place all ingredients into a small mixing bowl and whisk together. (this can be made up to 3 days in advance).
To assemble: Spread a small amount of the aioli onto each slice of brioche and top with butter lettuce, cucumbers and another drizzle of aioli. Add a few dashes of hot sauce, if using and serve with lemon wedges.