If you are on 8.1 or higher you can also use multitask view to close Apps that are open and you don't need them to be.
Hold the back button and enter card view. Scroll through the cards (which are open Apps) and press the X icon in the upper right of each App you want to close...