An urban low-height barrier meant to attenuate tramway noise for nearby walking pedestrians or cyclists is considered. The efficiency of this type of device is known to depend on the shape of the cross section and the acoustic properties of the surface treatment. Some sort of absorptive material is often required to enhance the performance by preventing the multi-reflection phenomenon, however such materials can be costly compared to acoustically rigid materials such as concrete. In this study, a rigid barrier is assumed but its shape is optimized using a sensitivity-based shape optimization algorithm coupled to the two dimensional BEM. The shape is here described in a very general fashion by mesh nodes coordinates, which can involve a large number of variables. Sensitivities with respect to all coordinates are calculated efficiently using the adjoint state approach, without significant increase of computation time. Numerical results show that optimized shapes tend to be quite irregular but provide a significant improvement compared to simpler shapes, especially in the mid and high frequency range. Intensity calculations seem to suggest that this improvement is due to scattering of the incident acoustic energy in the upwards direction, therefore reducing the diffiracted energy which reaches the shadow zone. Extra calculations show that the benefit of the optimized shapes can still be significant even in more realistic situations.