Purpose

The statistical similarity between synthetic data and real data refers to the degree by which the distributional properties of synthetic data match those of real data. It involves comparing various statistical characteristics such as mean, variance, correlation, and higher-order moments between real and synthetic datasets.

The goal of achieving statistical similarity is to ensure that synthetic data captures the underlying patterns, trends, and relationships present in real data. If synthetic data closely resemble real data in terms of its statistical properties, it is more likely to be useful and applicable for analytical tasks, modeling purposes, and downstream use cases.

When evaluating the statistical similarity between synthetic and real data, several aspects need to be considered. These aspects provide insights into the different properties and characteristics of the datasets. Here are the key aspects that need to be checked:

Univariate Level: These are your corner and edge pieces. They provide the basic structure of your dataset by analyzing one variable at a time. Each univariate metric within this level is like a different tool that helps you analyze each piece. For example, one tool (metric) might tell you the range of ages, another might tell you the most common gender in your dataset. When comparing real and synthetic data, these metrics help ensure that the basic structure of the synthetic dataset matches that of the real one. If the real data has most ages in the range of 0-25, the synthetic data should also reflect the same distribution.

Bivariate Level: These are the pieces that start to fill in the picture. They analyze the relationship between two variables. Within this level, different bivariate metrics can help you understand different aspects of the relationships. For instance, one metric might show you how age and gender interact in your dataset, like if there's a majority gender within a certain age group. When comparing real and synthetic data, these metrics ensure that relationships between pairs of variables in the synthetic data match those in the real data. If in the real data, 80% of individuals under the age of 25 are female, the synthetic data should maintain this relationship.

Multivariate Level: These are the intricate pieces that complete the puzzle. They take into account multiple variables at a time, offering a more complex, nuanced understanding of your data. Again, different multivariate metrics within this level can offer different perspectives on the interactions among the variables. For example, one could show how age, gender, and location interact. When comparing real and synthetic data, these metrics help ensure that the complex interactions among multiple variables in the synthetic data match those in the real data. If in the real data, most females under 25 from a certain location prefer a specific product, the synthetic data should capture this pattern.

By examining all these levels together, you can verify that your synthetic data is a faithful representation of your real data, much like how individual puzzle pieces come together to form a complete picture. The univariate level verifies the basic structure, the bivariate level checks the relationships between pairs of variables, and the multivariate level ensures that complex interactions among multiple variables are preserved. Together, they provide a full, detailed comparison between your real and synthetic datasets.

Fidelity Score (F)	Wasserstein Distance (WD)	Expected Fidelity
F > 95	WD < 0.05	Very High
80 < F < 95	0.05 < WD < 0.2	High
65 < F < 80	0.2 < WD < 0.35	Medium
F < 65	WD > 0.35	Low

Jensen-Shannon Distance (JSD) Banding

Fidelity Score (F)	Jensen-Shannon Distance (JSD)	Fidelity
F > 95	0	Very High
80 < F < 95	0.05 < JSD < 0.2	High
65 < F < 80	0.2 < JSD < 0.35	Medium
F < 65	JSD > 0.35	Low

Chi-squared Distance Banding

Fidelity Score	Chi-squared Distance	Fidelity
F > 95	CSD > 0.95	Very High
80 < F < 95	0.8 < CSD < 0.95	High
65 < F < 80	0.65 < JSD < 0.8	Medium
F < 65	JSD < 0.65	Low

All bivariate and multivariate are visual metrics. The following section explains how users can interpret visual metrics for understanding fidelity.

Statistical Similarity Summary

Purpose#

Purpose