The voltage noise of primary batteries with Li/MnO2 chemistry increases when the battery suffers a short circuit. This noise level is measurable only after a proper offsetting scheme that involves a battery that has undergone the same history is employed. In the current work, we are showing that the increased voltage noise is due to the metallic lithium anode and the heterogeneous discharge thereof. Periods of short circuit causes the lithium to get depleted from points of lowest resistance, which eventually causes uneven depletion and the current collector to be partially exposed. This is similar in nature to pitting corrosion of metals and their protective films, a phenomenon which is also routinely characterized by electrochemical noise signals.