All over-the-counter medicines, even aspirin, are only available at an Apotheke (pharmacy): the German term Drogerie or drugstore is a shop for sundry items.
Apotheken are open during normal business hours, with those in train stations or airports open later and on weekends. Apotheken are plentiful, and there is invariably one within a few blocks. Every district has an emergency pharmacy that is open after hours. These are listed as Apotheken-Notdienst or Apotheken-Bereitschaftsdienst on the window of every pharmacy in town, often with directions for how to get there. Pharmacies will have a bell you must ring to enter. Most pharmacists in larger cities speak enough English to help. Some drugs have different names: acetaminophen—or Tylenol—is called Paracetomol.