Revolutionizing Epidemic Surveillance: A Federated Approach

The Challenge of Real-Time Outbreak Detection

Rapidly identifying disease outbreaks is crucial for effective public health intervention. Early warnings about emerging pathogens or resurgent epidemics empower authorities to minimize transmission and prepare healthcare systems. However, real-time surveillance faces significant hurdles, especially in decentralized data environments like the United States, where hospitals, labs, and government agencies often hold fragmented information.

Traditional surveillance relies on mandated reporting, which is slow, cumbersome, and reactive. Barriers to data sharing include privacy concerns, competitive pressures, and institutional reluctance, hindering effective and timely responses to public health emergencies.

Federated Surveillance: A Powerful and Private Solution

Federated epidemic surveillance offers a groundbreaking alternative. This approach allows individual data custodians to retain control of their information. Instead of sharing raw data, they share only specific statistics, like p-values from hypothesis tests.

By aggregating these statistics, public health officials can detect trends indicative of potential outbreaks. This collaborative approach leverages the power of collective data without compromising individual privacy. Think of it as harnessing the wisdom of the crowd while respecting individual voices.

Imagine tracking COVID-19 hospitalizations across multiple facilities. Individual trends may be unclear, but pooling statistical insights reveals the larger picture – a sudden surge demanding immediate attention.

Methodology: Simple Yet Effective

Our research demonstrates the feasibility of federated surveillance using straightforward, easily explainable methods. We employ a two-step process: individual hypothesis tests for surges at each site, followed by meta-analysis to combine p-values into a single outbreak test.

We evaluated this approach using two real-world datasets: COVID-19 hospitalization data and insurance claims data related to COVID-19. These datasets represent different data characteristics – hospitalization data with larger counts from fewer sites and claims data with smaller counts from numerous sites.

Results: Matching Centralized Performance with Decentralized Data

Our findings reveal that federated surveillance can achieve performance comparable to centralized data analysis, the gold standard where all information is pooled. This is a remarkable achievement, showcasing the potential of this privacy-preserving approach.

In analyzing hospitalizations, Stouffer's method for combining p-values achieved impressive accuracy, detecting surges with high recall while maintaining a low false discovery rate. For insurance claims, Fisher's method proved most effective.

Crucially, using data from only the largest site consistently underperformed the federated approach, emphasizing the value of combining insights from multiple sources.

Optimizing Performance with Auxiliary Information

Federated surveillance can be further enhanced using readily available auxiliary information. Incorporating weights based on the relative contribution of each site – estimated from even infrequently updated total counts – significantly improves accuracy.

Our research shows that weighted versions of both Stouffer's and Fisher's methods outperform their unweighted counterparts and even surpass using data solely from the largest provider, even when the dataset is highly imbalanced.

The Future of Epidemic Surveillance

Federated surveillance represents a promising step towards modernizing public health infrastructure. By empowering data sharing without compromising privacy, this approach unlocks the full potential of decentralized data, enabling timely and effective responses to current and future health threats.