Federated Learning Excludes Rare Disease Research Sites with Small Datasets

The article discusses how the architectural limitations of federated learning exclude rare disease research sites with small patient populations, hindering progress in this critical field.

Details

The article highlights the architectural limitations of federated learning that prevent it from effectively supporting rare disease research. Rare diseases, by definition, affect a small number of patients, with 95% of the 7,000+ known rare diseases having no FDA-approved treatment. This is partly due to the economic challenges of conducting clinical trials for small patient populations. However, the article argues that there is also an architectural problem that the Orphan Drug Act did not address. Federated learning, the current standard for privacy-preserving distributed machine learning, requires each participating node to compute a gradient update from its local dataset. The mathematical requirement for gradient stability means that each node's local dataset must be large enough to compute a meaningful, low-variance gradient. For rare disease research sites that may only see 5-15 patients per year, this requirement is structurally unmet, and their data cannot be effectively utilized by the federated learning network. This 'N=1 problem' affects the majority of rare disease research, as over 6,650 of the 7,000+ known rare diseases impact fewer than 1 in 50,000 people. The article presents three scenarios illustrating how this architectural limitation excludes valuable data and expertise from rare disease research, hindering progress in this critical field. Addressing this challenge will require rethinking the fundamental architecture of distributed machine learning to better accommodate the realities of rare disease research.