Raw potatoes tend to be highly acidic. The average potato has a pH (a measure of acidity) of about 5. This is considered not highly acidic, but it is definitely far from neutral. The acid can be used in a chemical reaction to produce electricity, much in the same way a chemical battery functions.
In fact you could even make a potato battery using a piece of copper, a galvanized nail, some wire and a small lightbulb, preferably an LED, which requires little power and can be easily attached to the wires.
This same principle is largely used in synthetic batteries like a car battery.