pyspark.sql.functions.any_value

pyspark.sql.functions.any_value(col, ignoreNulls=None)

Returns some value of col for a group of rows.

Added in version 3.5.0.

Parameters:

colColumn or column name

target column to work on. A column of any type.

ignoreNullsColumn or bool, optional

if first value is null then look for first non-null value. A column that evaluates to a boolean. Must be a constant.

Returns:

Column

some value of col for a group of rows.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame(
...     [(None, 1), ("a", 2), ("a", 3), ("b", 8), ("b", 2)], ["c1", "c2"])
>>> df.select(sf.any_value('c1'), sf.any_value('c2')).show()
+-------------+-------------+
|any_value(c1)|any_value(c2)|
+-------------+-------------+
|         NULL|            1|
+-------------+-------------+

>>> df.select(sf.any_value('c1', True), sf.any_value('c2', True)).show()
+-------------+-------------+
|any_value(c1)|any_value(c2)|
+-------------+-------------+
|            a|            1|
+-------------+-------------+