Ref: Effective Java by Joshua Bloch
Strive to write good programs rather than fast ones. If a good program is not fast enough, its arhitecture will allow it to be optimized. Good programs embody the principle of information hiding: where possible, they localize design decisions within individiual modules, so individual decisions can be changed without affecting the remainder of the system.
Luckily, it is generally the case that good design is consistent with good performance. It is very bad idea to warp a design to achieve good performance.