pyspark.sql.functions.get#

pyspark.sql.functions.get(col, index)[source]#

Array function: Returns the element of an array at the given (0-based) index. If the index points outside of the array boundaries, then this function returns NULL.

Added in version 3.4.0.

Parameters:

colColumn or str: Name of the column containing the array. A column that evaluates to an array.
indexColumn or str or int: Index to check for in the array. A column that evaluates to an integer.

Returns:

Column: Value at the given position. Returns a column of the element type of the input array.

See also

pyspark.sql.functions.element_at()
pyspark.sql.functions.try_element_at()

Notes

The position is not 1-based, but 0-based index. Supports Spark Connect.

Examples

Example 1: Getting an element at a fixed position

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
>>> df.select(sf.get(df.data, 1)).show()
+------------+
|get(data, 1)|
+------------+
|           b|
+------------+

Example 2: Getting an element at a position outside the array boundaries

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
>>> df.select(sf.get(df.data, 3)).show()
+------------+
|get(data, 3)|
+------------+
|        NULL|
+------------+

Example 3: Getting an element at a position specified by another column

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index'])
>>> df.select(sf.get(df.data, df.index)).show()
+----------------+
|get(data, index)|
+----------------+
|               c|
+----------------+

Example 4: Getting an element at a position calculated from another column

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index'])
>>> df.select(sf.get(df.data, df.index - 1)).show()
+----------------------+
|get(data, (index - 1))|
+----------------------+
|                     b|
+----------------------+

Example 5: Getting an element at a negative position

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(["a", "b", "c"], )], ['data'])
>>> df.select(sf.get(df.data, -1)).show()
+-------------+
|get(data, -1)|
+-------------+
|         NULL|
+-------------+