pyspark.pandas.groupby.SeriesGroupBy.describe#

SeriesGroupBy.describe()[source]#

Generate descriptive statistics that summarize the central tendency, dispersion and shape of a dataset’s distribution, excluding NaN values.

Added in version 4.3.0.

Note

Unlike pandas, the percentiles in pandas-on-Spark are based upon approximate percentile computation because computing percentiles across a large dataset is extremely expensive.

Returns:

DataFrame: Summary statistics for each group and the series values.

See also

DataFrame.describe

Examples

>>> df = ps.DataFrame({'a': [1, 1, 3], 'b': [4, 5, 6]})
>>> df.groupby('a')['b'].describe().sort_index()
   count  mean       std  min  25%  50%  75%  max
a
1    2.0   4.5  0.707107  4.0  4.0  4.0  5.0  5.0
3    1.0   6.0       NaN  6.0  6.0  6.0  6.0  6.0