In federated representation learning (FRL), with the assistance of a central server, a group of N distributed clients train collaboratively over their private data, for the representations (or embeddings) of a set of entities (e.g., users in a social network). We develop SecEA, the first fully privacy-preserving framework for FRL. SecEA carries out lossless aggregation for each entity while simultaneously providing provable privacy guarantees for the set of entities and the corresponding embeddings at each client simultaneously, against a curious server and up to T N/2 colluding clients. As the first step of SecEA, the FRL system performs a private entity union protocol for each client to learn all the entities in the system without knowing local entity sets. In each aggregation round,...[ Read more ]

In federated representation learning (FRL), with the assistance of a central server, a group of N distributed clients train collaboratively over their private data, for the representations (or embeddings) of a set of entities (e.g., users in a social network). We develop SecEA, the first fully privacy-preserving framework for FRL. SecEA carries out lossless aggregation for each entity while simultaneously providing provable privacy guarantees for the set of entities and the corresponding embeddings at each client simultaneously, against a curious server and up to T < N/2 colluding clients. As the first step of SecEA, the FRL system performs a private entity union protocol for each client to learn all the entities in the system without knowing local entity sets. In each aggregation round, the local embeddings are privately shared among the clients using Lagrange interpolation, and then each client constructs coded queries to retrieve the aggregated embeddings for the intended entities. We perform comprehensive experiments on various representation learning tasks to evaluate the utility and efficiency of SecEA, and empirically demonstrate that compared with embedding aggregation methods without (or with weaker) privacy guarantees, SecEA incurs negligible performance loss, and the additional computation latency of SecEA diminishes for training deeper models on larger datasets.

Keywords: Federated Representation Learning, Secure Embedding Aggregation, Entity Privacy, Embedding Privacy, Private Information Retrieval.