A New York-based federal appeals court considered a closely watched case over transgender girls’ participation in female competition categories for school sports.