Privacy-preserving data normalization techniques with multiparty homomorphic encryption for federated learning

Abstract

Data normalization is a crucial preprocessing step for enhancing model per formance and training stability. In federated learning (FL), where data remains distributed across multiple parties during collaborative model training, normal ization presents unique challenges due to the decentralized and often heteroge neous nature of the data. Traditional methods rely on either independent client side processing, i.e., local normalization, or normalizing the entire dataset before distributing it to parties, i.e., pooled normalization. Local normalization can be problematic when data distributions across parties are non-IID, while the pooled normalization approach conflicts with the decentralized nature of FL. In this the sis, we explore the adaptation of widely used normalization techniques to FL and define the term federated normalization. Federated normalization simulates pooled normalization by enabling the collaborative exchange of normalization parameters among parties. Thus, it achieves performance on par with pooled normalization without compromising data locality. However, sharing normaliza tion parameters such as the median introduces potential privacy risks, which we further mitigate through a robust privacy-preserving solution. Our contributions include: (i) We systematically evaluate the impact of various federated and local normalization techniques in non-IID FL scenarios, (ii) We propose a novel ho momorphically encrypted k-th ranked element (and median) calculation tailored for the federated setting, enabling secure and efficient federated normalization, (iii) We propose privacy-preserving implementations of widely used normalization techniques for FL, leveraging multiparty fully homomorphic encryption (MHE).