To remove a background from a page, ensure that you do not have any items selected. You can make sure of this by clicking on an empty part of the page where only the background is visible. Then, either press the DELETE key on your keyboard or click on the Delete button in the second-level toolbar.
However, it is not necessary to delete a background before adding a new one. You can simply choose to drag and drop a new background on top of the old one. This will replace the old background with the new one.